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PREFACE 


The  Logistics  Research  Division  of  the  Armstrong  Laboratosy  (AL/HRG)  is  tasked  with 
understanding  the  impact  of  technology  on  people  and  organizations  as  new  information 
systems  move  from  the  laboratory  to  field  implementadon  in  System  Program  Offices  (SPOs) 
and  Air  Logistics  Centers  (ALCs).  In  1992,  the  Armstrong  Laboratory  created  an  initiative  in 
Human  Issues  in  Technology  Impiemeritation  to  understand  these  issues  and  develop  tools  and 
knowledge  that  would  support  SPOs  and  AIXs  in  their  efforts  to  adopt  new  technology. 

The  research  reported  here  was  an  anthropological  study  of  the  cultural  and  contextual  factors 
that  affect  the  receptivity  of  work  groups  in  AFMC  SPOs  to  new  information  technology.  The 
FRAMEAVORK  project  investigated  these  factors  and  developed  a  computer-based  readiness 
assessment  tool  for  their  assessment  in  the  SPO  environment.  The  research  was  carried  out  by 
Wizdom  Systems,  Inc.,  in  cooperation  with  Wayne  State  University,  under  contract  #  F41624- 
93-C-5016  from  the  Logistics  Research  Division  of  the  Armstrong  Laboratory.  The  principal 
investigator  was  Dr.  Allen  Batteau. 
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ft;!  Executi\^  Siimm^ 


This  report  presents  the  results  of  FRAMEAVORK;  Human  Issues  in  Information  Technolo2v 
Implementation,  a  Phase  11  Small  Business  Innovation  Research  Project  performed  by  Wizdom 
Systems,  Inc.,  under  contract  #  FA1624-93-C-5016  to  the  Lx)gistics  Research  Division  of  the 
Armstrong  Laboratory,  United  States  Air  Force,  AL/HRGA.  The  purpose  of  the  FRAMEAVORK 
project  was  first  to  identify  the  human,  organizational,  and  cultural  factors  that  impeded  or  facilitated 
the  implementation  of  information  technology  in  Air  Force  Materiel  Command  program  offices  and  Ar 
Logistics  Centers.  .Following  this,  the  second  purpose>of  the  project  was  to  develop  a  software  tool 
that  would  assist  SPQ,and  ALC  managers  in  addressing  these  issues. 

The  FRAMEAVORK  project  took  an  inductive  approach,  conducting  complete  cultural  assessments  of 
nine  SPO  and  ALC  components.  This  inductive  approach,  guided  by  current  research  in  human 
factors  and  sodotechnical  systems,  produced  results  tailored  to  the  needs  and  issues  of  the  Ar  Force 
Materiel  Command.  In  contrast  to  other  studies  testing  hypotheses  through  field  research,  the 
FRAMEAVORK  approach  concentrated  on  the  discovery  of  sociotechnical  issues  and  patterns  in  the 
AFMC.  From  this  discovery,  an  assessment  tool  ft)r  examining  different  organizations  and  a  set  of 
issue  reports  and  recommendations  for  managing  the  issues  were  developed. 

The  intent  of  the  FRAMEAVORK  project  lay  in  making  social  science  work  for  management  --  taking 
the  most  advanced  results  and  methods  of  the  social  sciences,  applying  them  in  ah  AFMC  context,  and 
developing  the  results  into  a  tool  that  would  be  useful  for  AFMC  managers. 

In  this  report,  we  first  present  the  theoretical  issues  underlying  the  adoption  and  implementation  of  new 
technologies  in  large,  complex  organizations.  These  issues  derive  from  a  substantial  body  of  research 
into  sociotechnologv  which  is  the  integration  of  social  systems  (consisting  of  groups,  roles,  statuses, 
and  networks)  with  technological  systems  (systems  for  managing,  transporting,  and  transforming 
material,  energy,  or  in  the  present  case,  information).  \Ve  have  taken  an  adaptive  approach  to  the  joint 
optimization  of  the  social  and  the  technological  system,  using  insights  and  findings  from  studies  of 
cultural  and  organizational  ecdlogy. 

We  next  present  our  conceptual  framework,  consisting  of  a  series  of  eleven  hypotheses  that  guided  our 
empirical  investigation.  Athough  our  approach  was  inductive  (pattern-finding,  rather  than  hypothesis- 
testing),  these  initial  guiding  ideas  provided  a  useful  structure. 

In  section  4  we  describe  this  inductive  approach,  detailing  the  methodology,  research  design,  sample 
issues,  scope  of  investigation,  operational  variables,  field  techniques,  and  coding  and  analysis.  The 
interview  protocol  for  the  field  study  is  contained  in  Appendix  B.  Attention  is  given  to  our  methods 
for  the  iterative  development  of  an  assessment  instrument  from  our  findings  at  the  field  sites. 
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Section  5  is  a  summary  of  our  field  research  and  other  development  activities,  including  software 
development  Additional  detail  on  software  development  is  given  in  appendices  C  and  D. 

In  section  6  we  present  our  findings  in  the  following  tenns:  those  that  have  broad,  general  application 
for  AFMC,  those  that  are  primarily  at  the  level  of  a  System  Program  Office;  and  intermediate  findings 
involving  issues  both  internal  and  external  to  the  SPOs.  Additional  detail  on  SPO  observ^ations  and 
results  is  contained  in  Appendix  E. 

Two  initiatives  were  taken  to  determine  the  feasibility  of  using  the  FRAIVIE/WORK  tool  in  USAF 
applications  development  and  implementation.  The  first  of  these  was  a  review  of  USAF  and  DoD 
policies  and  procedures  for  information  technology  development  and  implementation.  The  second  was 
a  specific  examination  of  how  FRAMEAVORK  could  support  the  deployment  of  JIMIS,  the  Joint 
STARS  Integrated  Maintenance  Information  System  at  the  depot  level.  The  results  of  both  of  these 
initiatives  are  presented  in  section  7. 

Wizdom  Systems,  Inc.,  has  received  numerous  inquiries  from  private  firms  expressing  interest  in  the 
capabilities  of  the  FRAMEAVORK  tool.  Currently,  FRAMEAVORK  is  being  adapted  to  two  new 
industries:  healthcare  informatics  and  the  automotive  industry.  Further  domain  development  of  the  tool 
is  anticipated  Wizdom's  commercialization  plans  for  FRAMEAVORK  are  described  in  section  8. 

We  identified  several  promising  areas  for  further  investigation  and  tool  development  in  the  course  of 
this  research.  These  include  an  improved  understanding  of  the  management  (as  contrasted  to  user) 
culture  that  bears  on  the  use  of  information  technology,  and  the  integration  of  readiness  assessment 
tools  such  as  FRAMEAVORK  with  network-based  systems.  Our  conclusions  and  recommendations 
are  described  in  section  9. 

The  FRAMEAVORK  project,  and  the  Armstrong  Laboratory  Human  Issues  in  Technology 
Implementation  initiative  that  supported  it,  have  created  for  the  U  S  Air  Force  an  important  new 
perspective  on  adopting  and  using  information  systems.  This  perspective  complements  and  improves 
upon  other  DoD  and  USAF  initiatives,  including  the  Continuous  Acquisition  and  Lifecycle  Support 
(CALS)  program,  the  Paperless  Acquisition  Initiative  (PAI),  Corporate  Information  Management 
(CIM),  the  Base  Level  System  Modernization  (BLSM)  project,  and  HRGA's  new  program  in 
Computer-Aided  Business  Engineering  (CABE).  The  integration  of  the  tools,  methods,  and  insights  of 
FRAMEAVORK  into  these  programs  and  initiatives  will  result  in  a  far  more  effective  and  cost-saving 
use  of  information  technology  by  the  U.  S.  Air  Force. 
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1 .  Statement  of  the  Problem 


The  FRAMEAVORK  project  was  a  Phase  II  Small  Business  Innovation  Research  (SBER)  project 
undertaken  by  Wizdom  Systems,  Inc.,  in  cooperation  with  Wayne  State  University.  Its  purpose  was  to 
develop  an  understanding  of  the  effects  of  U  S.  Air  Force  and  Program  Office  culture  and  human 
issues  on  the  implementation  of  advanced  information  technology,  and  make  those  findings  accessible 
in  a  form  that  would  be  useful  to  AFMC  management. 

The  Logistics  Research  Division  of  the  Armstrong  Laboratory  (AL/HRG)  is  tasked  with  understanding 
the  impact  of  technology  on  people  and  organizations  as  new  information  systems  move  from  the 
laboratory  to  field  implementation  in  System  Program  Offices  (SPOs)  and  Air  Logistics  Centers 
(ALCs).  In  1992,  the  Armstrong  Laboratory  created  an  initiative  in  Human  Issues  in  Technology 
Implementation  to  understand  these  issues  and  develop  tools  and  knowledge  that  would  support  SPOs 
and  ALCs  in  their  efforts  to  adopt  new  technology.  The  objectives  of  this  initiative  were  to: 

Define  the  domain  and  identify  critical  human  impacts; 

Provide  assessment  methods  for  measuring  these  impacts; 

Provide  systematic  means  to  determine  automation  impacts,  predict  cost/benefit  and 
success/failure,  guideline  for  implementation. 

In  recent  years,  the  military  has  spent  billions  of  dollars  on  information  systems,  yet  still  faces  the 
challenges  of  reducing  paperwork  and  achieving  efficient  communication  among  components. 
Information  systems  having  superior  technical  qualities  are  often  rejected  or  ignored  by  users.  Some  of 
the  sources  of  resistance  to  new  technology  include  the  complexity  and  difficulty  of  the  use  of  some 
systems,  and  the  loss  of  control  or  usefixlness  of  old  skills  as  new  systems  are  introduced. 

This  situation  is  not  unique  to  the  U  S.  Air  Force.  In  American  industry,  this  is  frequently  referred  to 
as  the  "productivity  paradox",  which  is  the  paradoxical  situation  that  despite  hundreds  of  biUions  of 
dollars  of  investment  in  information  technology,  industry  has  not  realized  commensurate  improvements 
in  output  per  man-hour.  Various  explanations  for  the  "productivity  paradox"  include  the  mis-alignment 
of  technology  and  business  methods  (Strassman),  the  failure  to  focus  on  process  (Hammer  & 
Champy),  and  the  lack  of  user  readiness  for  the  new  technology  (Adler). 

One  important  dimension  of  the  problem  is  that  human  and  cultural  factors  (HCF)  frequently  impede 
the  adoption  of  new  technologies.  Although  this  is  frequently  attributed  to  some  innate  "conservatism" 
or  "resistance  to  change"  that  is  supposedly  part  of  human  nature,  we  know  that  some  individuals  and 
groups  can  be  highly  innovative.  In  fact,  it  seems  more  plausible  to  attribute  both  innovativeness  and 
resistance  to  change  less  to  innate  human  nature,  and  more  to  both  the  cultural  background  and  the 
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immediate  context  in  which  people  find  themselves.  Some  cultures  are  known  to  be  highly  innovative, 
while  others  are  very  resistant  to  change.  (Figure  1)  Likewise,  some  individuals,  no  matter  how  open 
to  change  they  are  normally,  will  become  quite  averse  to  risk  in  certain  situations.  In  short,  to  the 
extent  that  we  can  model  both  culture  and  context  in  ways  that  are  meaningful  for  technological 
innovation,  we  can  create  a  tool  that  will  assist  the  manager  in  overcoming  the  cultural  and  contextual 
resistance  to  change. 


Cultural  factors  promoting 
inno^  ation:  flexible  social  structure, 
individualism,  pragmatic  orientation 


Context  factors 
inhibiting  innovation. 

homeostasis,  low 
levels  of  competition 


Innovative 

cultures 


Change- 

resistant 

cultures 


Context  factors 
promoting  innovation: 
ecological  diversitv', 
imperfect  adaptation 


Cultural  factors  inhibiting 
innovation:  rigid  stratification, 
religious  taboos 

Figure  1  —  Cultural  and  environmental  factors  promoting  innovation 


By  culture  we  understand  a  historically  evolving  tradition  of  practices  and  beliefs  that  are  uniquely 
shared  by  an  entire  group.  Because  individual  will  belong  to  multiple  groups,  he  or  she  will  participate 
in  multiple  cultures  and  subcultures:  one  can  speak  of  an  American  culture,  an  Air  Force  culture,  an 
AFMC  culture,  the  culture  of  a  single  SPO,  and  even  the  culture  of  work  groups  inside  the  SPO. 

By  context  we  understand  the  contingent  situation  of  an  organization  or  work  group.  Contextual 
features  can  include  both  the  physical  infi'astructure  and  the  budget  and  program  environment  and  the 
SPOs  relationship  with  other  programs  with  which  it  competes  for  fiands;  contextual  features  can  also 
include  the  world  diplomatic  situation;  when  the  American  armed  forces  are  on  full  alert,  the  work 
routines  inside  AFMC  change  perceptibly. 

In  the  next  two  sections  we  elaborate  on  our  use  of  this  concept.  Our  objective  was  to  develop  a 
method  and  a  tool  for  giving  the  SPO  or  ALC  manager  leverage  over  the  cultural  and  contextual  issues 
that  impeded  or  facilitated  technological  innovation  in  his  or  her  organization.  Our  development  of  the 
tool  rests  on  three  premises: 


Wizdom  Systems,  Inc. 


FRAMEAVORK  Final  Report,  page  5 


December  11,  1995 


•  Monitoring  and  measuring  are  management  functions.  The  idiom,  "li  you  measure  it, 
you  will  manage  it,"  expresses  important  insiglits  into  the  workings  of  management. 

•  A  tool  can  support  management,  but  it  is  no  substitute  for  management.  No  tool  can 
make  a  bad  manager  good;  the  right  tools  can  make  good  managers  better. 

•  An  important  class  of  management  tools  consists  of  those  that  collect,  organize,  and 
present  information  in  an  appropriate,  high-level,  focused  manner.  Examples  of  such 
tools  include  decision-support  databases,  spreadsheets,  and  forecasting  models. 

The  tool  we  envisioned  would  be  a  readiness  assessment  tool  with  which  a  SPO  or  ALC  manager 
could  pinpoint  the  human  issues  that  might  impede  the  adoption  of  new  systems  within  his  or  her 
organization.  We  have  chosen  the  tool  approach  as  an  alternative  to  a  printed  report  or  other  medium 
in  the  interests  of  the  widest  possible  dissemination.  We  have  chosen  to  embed  v/ithin  the  tool  an 
expert  system  that  would  capture  what  we  learned  from  fieldwork  in  SPOs  and  ALCs 

We  chose  an  empirical,  inductive  approach  for  building  this  tool.  Presently  there  is  no  standard 
language  or  variables  for  user  readiness  or  organizational  barriers  to  systems  implementation.  As  an 
alternative  to  asking  the  SPO  directors  and  MIS  managers  about  user  readiness  or  their  assessment 
tool  requirements,  we  chose  to  ask  the  users  about  the  issues  that  bore  on  their  readiness  to  begin  using 
new  systems.  This  connection  between  human  issues  and  technology  implementation  in  the  Air  Force 
has  been,  in  a  systematic  way,  uniquely  made  by  the  Armstrong  Laboratory  program  and  the  research, 
including  FRAMETWORK,  that  it  supported. 
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2.  Theoretical  Issues 


The  study  we  conducted  in  developing  the  FRAMEAVORK  tool  was  an  empirical  study  guided  by  the 
conclusions  of  numerous  other  studies  of  the  interaction  of  human  issues  and  technological  systems. 
Synthesizing  these  studies  laid  an  important  groundwork  for  our  subsequent  field  investigation. 

Our  understanding  of  the  relationship  between  technology  and  culture  is  drawn  fi’om  two  related 
theoretical  traditions:  socio-technical  systems,  and  human  and  cultural  ecology.  Socio-technical 
systems  theory  (also  referred  to  as  STS)  provides  a  broad  conceptual  framework  for  thinking  about  the 
ways  in  which  technological  and  human  systems  influence  one  another.  Human  and  cultural  ecology 
theories  offer  an  approach  to  culture  that  recognizes  the  role  of  material  artifacts  (e.g.,  technology)  and 
the  physical  environment  in  shaping  shared  traditions  of  behavior  and  belief  (i.e.,  culture).  Both  of 
these  theoretical  traditions  are  closely  aligned  with  systems  theory  (von  Bertallanfy  1956),  and  they 
may  be  related  to  one  another  in  terms  of  general  systems  principles. 


Socio-Technical  Systems  Theory 


Socio-technical  systems  theory  and  methodology  have  been  under  development  in  the  United  States 
and  Western  Europe  since  the  1950s.  Thousands  of  firms  have  contributed  to  the  development  of  STS 
through  implementation  of  STS  principles  in  process  re-design  and  new  technology  implementation 
programs  (Taylor  and  Felton  1993). 

The  basic  theory  of  STS  holds  that  all  work  organizations  are  comprised  of  two  interdependent 
subsystems:  the  technical  subsystem  and  the  social  subsystem.  The  technical  subsystem  includes 
technology  (i.e.,  tools,  devices,  methods,  procedures  for  transforming  inputs  into  outputs)  and  work 
process  (i.e.,  the  sequence  of  steps  required  for  technological  transformation).  Classically,  the  social 
subsystem  included  the  individual  humans  in  the  work  organization  and  their  social  and  organizational 
roles  and  relationships. 

Empirical  research  in  Great  Britain  during  the  1940s  recogniz;ed  the  distinctive,  yet  interdependent, 
status  of  these  two  subsystems  (Trist  1981).  The  technical  and  social  subsystems  are  distinctive  in  that 
each  of  them  is  shaped  by  different  sets  of  natural  laws.  Technology  operates  in  accordance  with  the 
laws  of  physical  science,  while  humans  behave  in  ways  that  are  explained  by  the  psychological  and 
social  sciences. 

The  interdependency  of  these  two  subsystems  can  be  understood  by  thinking  about  what  would  happen 
if  one  or  the  other  subsystem  were  to  be  removed  from  the  work  organization.  Just  as  humans  cannot 
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perform  work  effectively  without  tools  and  sequential  methods  of  operation,  technologies  cannot 
fiinction  without  humans  and  organizational  structures  to  deploy,  operate  and  maintain  them.  Without 
people,  technology  becomes  inert  matter  eventually.  Several  implications  follow  from  the  observation 
that  the  two  subsystems  need  each  other  (i.e.,  are  interdependent).  One  implication  is  that  they 
influence  one  another  in  a  variety  of  ways.  For  example,  technology  can  shape  people's  physical 
behavior  (e  g.,  requiring  them  to  sit  at  a  work  station  and  manipulate  a  keyboard).  People  in  a  work 
organization,  on  the  other  hand,  can  sabotage  technology  either  overtly  or  covertly,  rendering  it 
useless. 


Implications  of  Sociotechnology 

The  observation  that  technical  and  social  subsystems  are  distinctive  yet  interdependent  has  significant 
implications  for  the  management  of  technology  in  work  organizations.  One  implication  is  that  an 
organization  cannot  simply  change  one  subsystem  (e.g.,  technology)  and  expect  that  subsystem 
to  perform  as  it  if  were  operating  under  laboratory  conditions.  Once  deployed  in  a  work 
organization,  a  new  technological  system  will  be  interdependent  with  the  social  subsystem  in  that 
organization,  meaning  that  the  social  subsystem  will  have  an  influence  on  the  fianctioning  of  the 
technology.  This  influence  can  either  enhance  or  inhibit  the  technology's  operation  (Majchrzak  1992). 


Another  implication  is  that  significant  change  in  one  subsystem  (i.e.,  technology)  often  will  require 
changes  in  the  other  subsystem  if  the  technology  is  to  operate  effectively.  The  social  subsystem  in 
a  work  organization  typically  is  adapted  or  adjusted  to  the  old  technology,  and  failure  to  alter  the  social 
system  as  the  technology  is  changed  can  lead  to  suboptimal  performance  of  the  new  technology  (Adler 
1989, 1990). 

A  third  implication  is  that  changes  in  either  the  technical  or  social  subsystems  of  a  work 
organization  must  recognize  and  accommodate  the  principles  of  both  physical  and 
psychological/social  sciences.  If  a  technology  change  program  attempts  to  optimize  the  performance 
of  the  technology  alone,  forcing  the  social  side  to  behave  in  accordance  with  physical  principles  and  not 
considering  the  needs  and  requirements  of  the  human/social  side  of  the  organization,  then  interference 
from  the  human/social  side  will  cause  the  work  organization  as  a  whole  to  exhibit  suboptimal 
performance. 

This  third  observation  jdelds  the  general  principle  of  joint  optimization,  which  states  that  the  work 
organization  should  be  designed  or  re-designed  through  mutual  adjustment  of  both  the  technical  and 
social  subsystems  (Trist  1981).  Optimal  performance  in  the  work  organization  as  a  whole  will  be 
achieved  when  the  needs  and  requirements  of  both  the  technical  and  social  subsystems  are  considered 
and  adjusted  to  fit  each  other,  rather  than  attempting  to  optimize  the  performance  of  either  the 
technical  or  social  sides  alone. 
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The  principles  of  STS  theory  suggest  that  technology  change  must  be  planned  carefully  to  include 
adjustments  in  the  social  side  of  the  organization.  New  technology  deployment  provides  an 
opportunity  for  re-design  of  the  work  organization,  meaning  a  re-configuration  of  technical  and  social 
subsystems.  This  implies  that  the  social  subsystem  should  be  well  understood  by  those  undertaking  the 
reconfiguration;  under  most  conditions  it  is  not  possible  to  re-design  a  social  subsystem  if  the 
properties  of  the  existing  subsystem  are  not  understood  (the  primary  exception  being  a  greenfield  site, 
and  even  there  the  social  backgrounds  of  participants  often  are  taken  into  account  (Stoffel  and  van 
Willigen  1986). 

Socio-technical  systems  theory  was  developed  in  the  1940s  and  1950s,  and  it  bears  the  intellectual 
hallmarks  of  ideas  fi'om  that  era.  Two  limitations  of  STS  theory  as  described  above  are  a)  it  does  not 
address  the  influence  of  the  environment  that  surrounds  the  socio-technical  system,  and  b)  it  does  not 
include  an  explicit  recognition  of  the  role  of  ambient  cultures.  Human  and  cultural  ecology  embody  a 
set  of  ideas  that  address  these  limitations,  and  add  a  number  of  additional  concepts  that  are  very  useful 
in  understanding  the  interaction  of  technology  and  people. 


Human  and  Cultural  Ecology 

The  science  of  ecology  is  concerned  with  the  relationships  between  living  organisms  and  their 
environments.  Human  and  cultural  ecology  extend  this  concern  to  inelude  human  social  systems  — 
human  populations  (such  as  work  organizations)  become  the  focus  of  investigation,  both  in  terms  of 
their  relationships  with  the  physical  environment,  and  with  other  elements  of  the  human  environment 
(e  g.,  other  cultures). 

Cultural  ecology  is  particularly  concerned  with  the  connection  between  culture-environment  relations 
on  the  one  hand,  and  the  way  in  which  these  relations  influence  the  interaction  of  elements  within  a 
sociocultural  system  on  the  other  (Steward  1955,  Ortner  1984).  The  internal  patterning  of  a  cultural 
system  is  viewed  as  the  principal  mechanism  through  which  humans  respond  to  environmental 
opportunities  and  constraints.  From  this  perspective,  culture  is  defined  as  a  system  of  habitual 
behavioral  practices  and  a  related  set  of  assumptions,  beliefs,  and  knowledge  that  emerge  through  a 
social  group's  interaction  with  its  environment. 

So  defined,  culture  represents  an  additional  dimension  of  the  social  subsystem  within  an  integrated 
socio-technical  system,  together  with  individual  humans  and  their  social  relationships.  Cultural  patterns 
that  emerge  through  time  as  a  result  of  environmental  interaction  will  influence  the  deployment  of  new 
technology  in  a  work  organization.  Thus,  cultural  principles  must  be  considered  in  planning  for 
technology  change. 

Formal  work  organizations  may  be  viewed  as  cultures  (or  subcultures)  when  they  share  a  common  goal 
and  a  common  set  of  work  tasks,  interact  regularly  over  long  periods  of  time,  and  develop  shared 
practices  and  beliefs  pertaining  to  their  work  (Baba  1995),  In  this  study  for  the  Armstrong  LaWatory, 
we  conceptualized  the  entire  U.S.  Air  Force  (USAF)  as  a  cultural  entity  or  population,  and  also 


Wizdom  Systems,  Inc. 


FRAMEAVORK  Final  Report,  page  9 


December  11,  1995 


viewed  various  AF  subunits  (e.g  ,  SPOs  or  three  letters)  as  subcultures  if  they  displayed  a  lengthy 
historical  tradition  and  shared  patterns  of  behavior  and  belief 


The  Concept  of  Environment 

Environment  is  an  ambiguous  concept  that  may  be  defined  in  different  ways  from  different  points  of 
view.  Cultural  ecologists  have  attempted  to  define  the  effective  environment,  including  both  objective 
(etic)  and  subjective  (emic)  conditions  and  characteristics  of  the  cultural  context  that  appear  to  have  a 
bearing  on  group  practices  and  beliefs  (Manners  and  Kaplan  1972). 

An  important  element  of  environmental  analysis  is  the  identification  of  specific  ecological  niches  that 
are  inhabited  by  different  populations.  Niche  also  is  a  difficult  concept,  since  it  can  only  be  defined  with 
reference  to  the  behavior  of  its  occupant  (Hannan  and  Freeman  1989).  Generally  speaking,  a  niche 
consists  of  a  specific  set  of  environmental  resources  that  are  utilized  and  contributed  by  a  specific 
population  (Aldrich  1979).  In  organizational  ecology,  several  types  of  variables  (or  dimensions)  have 
been  utilized  to  define  niches,  including:  resource  abundance  and  scarcity;  environmental  homogeneity 
and  heterogeneity,  stability  and  instability,  and  concentration  and  dispersion;  degree  of  domain 
consensus;  and  degree  of  turbulence. 

In  classical  cultural  ecology,  the  environment  generally  has  meant  the  geographical,  biological,  and 
cultural  habitat  surrounding  a  whole  sociocultural  system  (e  g.,  a  tribe  or  village).  In  organizational 
studies  of  ecology,  on  the  other  hand,  the  environment  often  has  meant  the  external  context  of  the 
entire  organizational  system  (e  g.,  a  corporation's  market,  competitive,  or  regulatory  environment). 
Writers  also  have  characterized  the  internal  environment  of  a  complex  organization,  i.e.,  the 
environment  that  exists  inside  a  work  organization,  yet  is  external  to  the  subunits  within  the 
organization  (see  Burgleman  1991,  Baba  1995). 


The  cultural  environment  of  work  groups 

The  cultural  environment  of  any  given  work  group  subculture  has  a  number  of  components,  including 

•  Other  work  group  subcultures  within  the  organization 

•  Dynamic  interactions  among  these  work  groups  which  together  constitute  the  overall 
culture  of  the  organization 

•  Multiple  cultural  contexts  external  to  the  organization  which  have  an  influence  on  it 
(e.g.,  national,  regional,  or  occupational  cultures). 

A  comprehensive  analysis  of  a  work  group's  cultural  environment  includes  an  examination  of 
culture-environment  relationships  within  each  of  these  categories. 
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Since  our  work  for  the  Armstrong  Laboratories  focused  on  technology  deployment  in  specific  Air 
Force  Materiel  Command  (AFMC)  subunits,  within  the  context  of  the  acquisition  and  logistics 
processes,  one  logical  approach  to  conceptualizing  physical  and  cultural  environments  would  be  to  use 
the  internal  patterning  of  the  AF  acquisition  and  logistics  process  itself  as  a  framework  for  thinking 
about  the  effective  environment.  In  the  acquisitions  and  logistics  process,  new  products  pass  through 
five  broad  phases,  including: 

1)  concept  origination 

2)  engineering  design 

3)  engineering  development  and  testing 

4)  manufacturing 

5)  maintenance 

AFMC  is  attempting  to  integrate  these  phases  through  the  Integrated  Weapons  System  Management 
(fWSM)  doctrine.  Under  IWSM  all  phases  will  be  considered  simultaneously  as  the  process  unfolds. 
In  reality  many  formal  work  groups  concentrate  their  efforts  most  intensely  in  one  of  these  phases  (eg., 
engineering  design,  or  logistics).  For  this  reason,  we  conceptualized  the  environment  of  our  focal  units 
in  terms  of  the  acquisitions  and  logistics  process  and  its  inherent  stage-process  structure.  The  specific 
features  of  this  environment  at  various  stages  comprise  environmental  variables  that  influence  the 
development  of  work  group  subcultures.  These  sub-cultures,  in  turn,  influence  the  deployment  of  new 
technologies  within  the  environment.  Environmental  variables  specific  to  the  acquisition  and  logistics 
process  are  described  in  the  next  section  of  the  report. 


The  Concept  of  Adaptation 

A  central  concept  in  ecology  is  that  of  adaptation.  The  principal  idea  behind  the  ecologist's  use  of  the 
term  adaptation  is  that  the  population  responds  or  adjusts  to  changing  environmental  conditions 
(including  changes  introduced  by  other  sociocultural  systems)  in  a  way  that  enables  the  population  to 
maintain  or  enhance  those  relations  with  the  environment  that  are  requisite  to  the  population's 
continued  existence  and  well-being.  Adaptation  is  viewed  as  a  process  rather  than  an  end-state,  thus 
avoiding  the  tautology  that  is  created  when  adaptation  is  defined  as  survival  in  a  changing  environment, 
and  survival  is  then  offered  as  proof  of  adaptation. 

Human  ecology  (Hawley  1986)  provides  a  useful  addition  to  our  thinking  about  adaptation  by 
suggesting  that  communities  of  populations  collectively  adapt  to  environmental  conditions,  thus 
forming  an  interacting  ecosystem,  comprised  of  an  interdependent  network  of  populations  and  their 
environments.  According  to  Hawley  (1986),  an  ecosystem  is  an  arrangement  of  mutual 
interdependencies  among  units  in  which  the  group  of  units  operates  as  a  single  whole,  thereby 
maintaining  a  viable  relationship  with  the  environment. 

In  our  study,  we  viewed  the  distinctive  subcultures  of  any  given  AFMC  work  group  as  an  adaptive 
response  to  that  group's  environment,  created  by  a  pattern  of  interaction  and  interdependency  among 
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communities  or  networks  of  work  groups  in  which  a  focal  group  was  embedded.  Our  data  suggest 
that  distinctive  subcultures  form  as  a  response  to  interaction  patterns  among  work  groups  in  such 
ecosystems.  For  example,  the  cultural  differences  between  engineers  and  logisticians  are  due,  in  part, 
to  the  different  environmaits  in  which  they  work:  the  engineers  work  with  ideas  and  concepts  and 
images,  whereas  the  maintainers  work  with  metal  and  plastic  and  black  boxes;  the  engineers  rub  elbows 
with  program  managers  and  budget  oflScers  and  test  pilots;  the  maintainers  must  work  with  depot 
commanders  and  shop  technicians  and  pilots  whose  aircraft  they  have  grounded.  Although  there  are 
many  engineers  in  the  depots,  and  every  System  Program  Of5ce  has  a  logistics  function,  in  truth  the 
SPO  environment  is  a  development  environment,  whereas  the  depot  environment  is  a  support  and 
maintenance  environment.  Once  formed,  a  distinctive  subculture  then  plays  a  role  in  maintaining 
ecosystem  interaction  patterns  over  time.  These  principles  are  illustrated  in  later  discussion  of  our 
findings. 


The  Concept  of  Change 

Ecological  change,  defined  as  an  irreversible  and  non-repeatable  process  that  involves  alteration  of  an 
entire  ecological  system,  usually  results  fi-om  an  interaction  of  external  forces  and  internal  conditions 
(Hawley  1986).  Organizational  ecologists,  who  examine  adaptation  by  and  within  contemporary 
organizations  have  noted  that  organizations  often  display  inertia  in  the  fece  of  environmental  change 
(Hannan  and  Freeman  1989).  Selection  appears  to  favor  reliable  performance  and  stability,  meaning 
^t  organizations  develop  routines  or  habitual  practices  that  are  highly  resistant  to  change.  An 
important  debate  in  the  literature  of  organizational  ecology  is  between  those  who  believe  that 
organizations  are  basically  inertial  (meaning  that  new  capabilities  arise  only  when  new  types  of 
organizations  are  bom),  and  those  who  hold  the  contrary  view  that  new  organizational  capabilities 
emerge  gradually  through  conscious  adaptation  of  existing  organizations  to  environmental  pressure 
(Hannan  and  Freeman  1989).  (In  contrast  to  cultural  ecology,  which  considers  primarily  a  group's 
adaptation  to  the  natural  environment,  organizational  ecology  is  more  concerned  with  the  social 
environment.) 

From  our  standpoint,  new  information  technology  being  deployed  within  AFMC  within  office  and 
engineering  environments  presents  not  only  an  effort  to  change  the  socio-technical  systems  of  AFMC 
work  groups,  but  also  reflects  a  significant  environmental  shift  which  challenges  the  existing  stmcture 
and  functioning  of  work  groups  and  their  ecosystems.  New  information  technology  and  other  change 
initiatives  such  as  IWSM  attempt  to  change  the  nature  of  work  group  boundaries  and  exchanges  by 
bringing  some  groups  into  close  connection  with  other  groups  that  traditionally  have  been  culturally 
distant  (e.g.,  design  and  logistics,  or  the  AF  and  external  contractors).  New  technology  deployment 
also  adds  significant  turbulence  to  the  environment. 

In  the  FRAMEAVORK  project  we  found  that  in  some  ecosystems,  work  groups  responded  to  this  shift 
by  vigorous  efforts  to  maintain  the  status  quo  ante  (i.e.,  resisting  technological  change),  while  others 
responded  by  participating  actively  in  the  process  of  new  technology  adoption  and  implementation 
(thereby  transforming  themselves).  A  key  goal  of  our  analysis  was  to  gain  a  better  understanding  of 
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the  factors  and  forces  that  played  a  role  in  shaping  these  two  divergent  types  of  responses  to 
environmental  change. 
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Figure  2  -  Sociotechnology  and  Cultural  Ecology 


Integrating  Socio-Technical  System  Theory  with  Human  and  Cultural  Ecology 

Bringing  together  STS  and  human/cultural  ecology  theory  makes  each  of  these  approaches  more 
robust.  STS  theory  is  enhanced  by  a)  the  addition  of  an  open  systems  perspective  (i.e.,  an 
understanding  that  socio-technical  systems  interact  with  their  environments),  and  b)  the  enrichment  of 
the  social  side  through  inclusion  of  its  cultural  dimension.  Cultural  ecology,  on  the  other  hand,  gains 
from  the  STS  focus  on  social  roles,  relationships  and  structure  that  was  missing  from  classical 
approaches  to  cultural  ecology  (Ortner  1986).  The  process  of  further  integrating  these  two 
theoretical  traditions  is  an  on-going  challenge  which  is  underway  at  Wayne  State  University's 
Laboratory  for  Socio-Technical  Systems  Integration.  The  laboratory  recently  received  a  major 
three-year  grant  (1995  -  1997)  from  the  National  Science  Foundation  to  continue  its  efforts  at 
conceptual  integration. 

At  this  stage,  the  conceptual  advances  that  have  been  achieved  from  integration  efforts  include  the 
following  understandings; 
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1)  Socio-technical  systems  do  not  exist  in  a  vacuum,  but  are  part  of  a  larger  environmental 

whole  which  includes  both  physical  and  cultural  elements; 

2)  The  social  subsystem  of  a  work  organization  has  psychological,  sociological,  and  cultural 

dimensions,  the  latter  defined  as  an  historically  evolving  tradition  of  behaviors  and 
beliefs  formed  as  a  response  to  emdronmental  interactions; 

3)  Technology  change  is  influenced  by  the  psychological,  sociological,  and  cultural  dimensions 

of  a  work  organization; 

4)  Socio-technical  systems  (including  their  cultural  dimension)  exist  simultaneously  at  several 

diiBferent  ^et  related)  levels  of  analysis,  including  the  level  of  the  work  group  (e  g.,  a 
branch  or  division),  the  larger  organization  in  which  a  work  group  is  embedded  (e.g.,  a 
SPO),  and  the  macro-environmental  level  of  the  surrounding  context  (e  g  ,  the  United 
States,  the  Air  Force,  occupational  culture  of  engineering); 

5)  Connections  among  socio-cultural  entities  (e  g.,  informal  linkages  between  different  work 

group  subcultures,  or  personal  orientations  that  connect  different  cultural  contexts) 
may  be  one  of  the  ways  in  which  socio-technical  systems  at  different  levels  of  analysis 
are  integrated.  In  this  same  vein,  behavioral,  cognitive,  and  intellectual  boundaries 
between  cultural  entities  inhibit  potential  linkages  across  units  that  could  be  facilitated 
by  technological  means  (as  will  be  illustrated  repeatedly  in  the  discussion  of  our 
findings). 
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3.  Conceptual  Framework 


In  developing  the  FRAMEAVORK  tool  we  took  an  inductive  approach  to  understanding  the 
interaction  of  culture  and  technology  in  AFMC;  we  were  guided  in  this  by  a  conceptual  framework  and 
a  set  of  hypotheses  describing  possible  relationships  between  work  groups,  their  relationship  to  their 
environment,  and  their  acceptance  of  new  information  technology.  Rather  than  simply  testing  a  set  of 
hypotheses,  our  intent  was  to  take  ideas  and  results  from  our  research  and  previous  research,  and  make 
it  useful  to  AFMC  management. 

The  conceptual  framework  for  our  research  begins  from  the  assumption  that  work  organizations  are 
socio-technical  systems  (in  the  terms  described  in  the  previous  section).  This  means  that  an 
understanding  of  the  fectors  and  forces  that  influence  the  deployment  and  implementation  of  new 
information  technology  must  include  an  investigation  of  the  psychological,  sociological,  and  cultural 
properties  of  the  work  organization  (i.e.  the  social  subsystem),  and  the  nature  of  interdependencies 
between  these  properties  and  the  technical  subsystem.  An  approach  that  examines  only  one  of  these 
dimensions  (e.g.,  only  the  psychological  characteristics  of  individuals)  will  M  to  identify  other 
significant  factors  (e  g.,  those  emerging  from  the  sociological  context,  including  those  related  to  the 
shared  patterns  of  behavior  and  belief,  or  culture).  We  assume  that  the  AS-IS  social  subsystems  in 
various  organizational  units  (including  psychological,  sociological,  and  cultural  properties)  may  act  as 
facilitators  of  technology  change,  or  they  may  inhibit  such  change. 

A  second  assumption  that  underpins  our  conceptual  framework  is  the  notion  that  the  socio-technical 
system  forms  in  part  as  a  response  to  environmental  opportunities  and  constraints.  Thus,  in  seeking 
regularities  across  organizational  subunits  in  responses  to  technology  change  (e.g.,  adoption  versus 
rgection  of  new  technology),  we  must  examine  pre-existing  environmental  properties  and  look  for 
ways  in  which  these  properties  are  linked  to  regularities  in  socio-technical  systems.  If  certain 
environmental  properties  are  regularly  associated  with  certain  types  of  socio-technical  systems  (i.e., 
those  that  regularly  resist  or  accept  change),  then  we  may  be  able  to  predict  the  location  of 
socio-technical  barriers  and  plan  for  their  management  or  elimination. 

Our  exploratory  Phase  I  research  was  aimed  at  identifying  independent  variables  related  to 
environments,  and  to  properties  of  existing  socio-technical  systems  (i.e.,  work  groups),  that  might  be 
regularly  linked  to  our  primary  dependent  variable  (i.e.,  implementation  or  non-implementation  of  new 
information  technologies).  This  early  research  identified  several  independent  variables  that  held  the 
potential  to  explain  differences  in  work  group  responses  to  technology  change.  Systematic  data 
gathering  in  Phase  n  confirmed  or  refuted  the  significance  of  these  early  variables,  and  brought  other 
potentially  significant  variables  to  our  attention.  These  variables  all  emerged  out  of  the  field  study  of 
eleven  different  AFMC  components.  The  importance  of  this  is  in  understanding  user-level  issues.  All 
of  these  environmental  and  socio-technical  variables  (described  below)  may  be  understood  within  the 
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context  of  our  theoretical  framework.  Later  discussion  of  our  research  findings  identifies  those 
variables  that  were  supported,  and  those  that  were  refuted. 


empirical 
research  & 
findings 


Figure  3  -  Development  of  hypotheses  and  empirical  findings 


Independent  Variables  Related  to  the  Environment 


Stage  in  Acquisition/Logistics  Process.  Our  conceptualization  of  the  environment  suggests 
that  work  groups  may  be  located  along  a  stage-process  continuum  associated  with  various 
phases  of  the  acquisition/logistics  process.  Depending  upon  the  phase  in  which  a  group 
concentrates  its  eflforts,  the  physical  and  cultural  environment  of  the  group  will  differ  (Baba 
1995).  For  example,  groups  that  concentrate  on  engineering  design  will  have  intensive 
interactions  with  the  design  departments  of  contractor  organizations,  while  those  that 
concentrate  on  logistics  will  have  more  contact  with  flying  aircraft  and  their  crew. 

Hypothesis  #1 ;  differences  in  these  environments  may  stimulate  or  impede  work  group  interest 
in  new  information  technology  (e.g.,  since  AF  design  engineers  rely  on  designers  from 
contractor  organizations  to  be  knowledgeable  regarding  computer-aided  design  technology, 
the  AF  engineers  might  not  be  highly  motivated  to  take  an  interest  in  such  technology 
themselves). 

Volume  of  Paperwoit  Many  of  the  new  information  technologies  under  investigation  are 
designed  to  manage  the  volume  and  flow  of  paperwork.  We  reasoned  therefore  that  work 
groups  with  a  high  overall  volume  of  paper  under  their  control  might  be  interested  in  new 
information  technologies  with  support  paper  management.  This  hypothesis  follows  from 
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Rogers'  (1982;  Tomatzky  and  Fleischer  1990)  work  which  shows  that  innovations  which  are 
compatible  with  a  group's  needs  will  be  adopted  more  readily. 

Hypothesis  #2:  Since  the  volume  of  paper  tends  to  grow  over  the  life  of  an  aircraft  program, 
we  hypothesized  that  groups  farther  along  in  the  acquisition/logistics  process  (e.g.,  logistics 
groups)  might  be  more  highly  motivated  to  adopt  new  technology. 

Resource  Abundance  or  Scarcity.  Also  following  from  Rogers'  (1982)  observations,  we 
reasoned  that  new  information  technology  often  is  costly  and  also  represents  a  risk  (since  the 
payofis  are  unknown  at  the  point  of  adoption).  Therefore,  groups  with  available  slack 
resources  should  be  more  likely  to  invest  in  technologies  with  uncertain  outcomes. 

i 

Hypothesis  #3:  Larger  programs  with  reasonably  stable  funding  (e.g.,  the  B2  program)  should 
be  candidates  for  new  technology  adoption. 

Turbulence.  Turbulence  means  that  the  environment  is  changing  in  ways  that  are  not 
controlled  by  the  work  group,  and  that  changes  originating  in  distant  locations  disrupt 
operations  at  the  local  level,  often  without  warning  (Emery  and  Trist  19  ).  While  new 
information  technology  may  enable  the  work  group  to  become  aware  of  disturbances  before 
they  disrupt  operations,  the  very  fact  of  turbulence  sets  up  conditions  that  make  new 
technology  integration  very  difficult  (i.e.,  because  the  experts  and  champions  who  are  needed 
to  get  it  up  and  running  cannot  be  counted  on  to  remam  m  residence  over  the  duration  of  the 
implementation  period). 

Hypothesis  #4:  Ifrgher  degrees  of  turbulence  may  be  associated  with  fewer  successful 
examples  of  technology  integration  over  time. 

Supplier  Environment.  An  important  element  in  any  organization's  environment  is  the  nature 
of  other  organizations  that  surround  it,  and  with  which  it  interacts.  Relations  between  a  focal 
organization  and  other  organizations  in  its  environment  are  one  of  the  significant  fectors 
influencing  the  behavior  of  the  focal  organization  (Alrich  1979).  If  distrust  exists  between  one 
organization  and  another,  then  there  could  be  barriers  to  the  flow  of  information  between  them 
(Baba  1995).  Because  information  is  a  resource  that  can  be  used  either  as  a  means  to  gain 
advantage  for  an  organization,  or  to  harm  an  organization,  access  to  vital  information  often  is 
protected  from  distrusted  outsiders.  Trust  includes  confidence  in  the  othef  s  competence  in  his 
intentions. 

Hypothesis  #5:  Relations  of  trust  or  distrust  between  the  Air  Force  and  its  suppliers  influence 
the  ease  with  which  electronic  connections  are  established  between  them,  and  the  extent  to 
which  these  connections  are  used. 

Air  Force  Culture.  As  our  fieldwork  progressed  our  understanding  of  Air  Force  and  AFMC 
culture  increased,  leading  to  an  understanding  of  additional  critical  variables.  The  cultural 
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characteristics  of  the  US  Air  Force  represent  an  important  dimension  of  the  cultural, 
environment  of  the  SPOs  and  ALCs  that  are  adopting  information  technology.  Characteristics 
of  this  cultural  environment  could  influence  technology  adoption  and  use,  or  rejection,  in  a 
variety  of  ways.  For  example,  the  Air  Force  appears  to  view  technology  as  the  solution  to 
many  of  its  problems  (as  evidenced  by  the  heavy  reliance  on  advanced  weapon  systems  in 
recent  hostile  engagements).  When  managers  believe  that  technology  is  a  "silver  bullet"  that 
can  be  "thrown  at"  difficult  problems  with  the  expectation  of  automatic  or  immediate  results, 
there  is  a  danger  of  certain  misconceptions  which  can  place  technology  deployment  at  risk  (see 
Baba  et  al  1995).  One  of  these  misconceptions  is  the  notion  that  you  can  "plug  it  in  and  forget 
it",  without  the  need  for  careful  adjustments  in  skills,  process,  and  structure  that  often  are 
required  to  get  the  maximum  benefit  out  of  new  technology  (Adler  1989).  Another  element  of 
Air  Force  culture  that  influences  technology  adoption  is  the  contrast  between  patterns  of 
mobility  in  the  military  and  civilian  services.  Military  executives  change  positions  more  rapidly 
than  civilians,  and  this  movement  may  represent  a  disruptive  force  with  respect  to  new 
technology  deployment  efforts,  particularly  if  military  executives  are  the  primary  drivers  of 
change. 

Hypothesis  #6:  Units  closely  associated  with  these  mission-critical  activities  (e.g.,  engineers 
who  develop  fighter  engines)  may  not  be  interested  in  new  information  technology  unless  it 
directly  benefits  the  primary  mission  (otherwise,  it  may  be  viewed  as  uninteresting  or 
unimportant). 


Independent  Variables  Related  to  Properties  of  Socio-Technical  Systems 


In  addition  to  these  variables  derived  from  cultural  ecology,  we  have  a  set  of  variables  and  hypotheses 
derived  from  the  sociotechnical  systems  theory.  Again,  the  hypotheses  presented  here  were  less 
hypotheses  that  we  were  testing  than  they  were  ideas  that  guided  our  empirical  investigation: 

Pre-Existing  Technology  Use.  The  existing  technology  utilized  by  a  work  group  is  an  integral 
component  of  the  group's  AS-IS  socio-technical  system.  This  means  that  the  group  probably 
has  in  places  social  subsystem  elements  that  enable  it  to  utilize  the  technology  that  is  already 
present. 

Considered  as  hardware  and  software,  legacy  technology  is  a  technical  variable.  Considered  in 
terms  of  attitudes  and  user  roles,  legacy  systems  are  an  important  sociotechnical  variable. 
Attitudes  are  particularly  revealing,  inasmuch  as  they  reflect  both  enduring  values  (culture)  and 
current  use  patterns  (context). 
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Hypothesis  #7;  The  more  advanced  the  existing  technology,  the  more  likely  it  is  that  other 
advahced  technologies  will  be  compatible  with  existing  social  system  elements,  and  thus  more 
likely  to  be  integrated  successfully. 

Hypothesis  #8:  Positive  user  attitudes  toward  existing  computer  systems  will  support  the 
implementation  of  new  systems. 

Organizational  Structure,  fragmentation,  size,  and  type.  Work  groups  (or  other 
organizations)  that  display  many  internal  boundaries  (e.g.,  a  functional  division  of  l^or,  with 
occupational  "chimneys"  that  communicate  vertically  but  not  horizontally)  will  be  incompatible 
with  information  technologies  that  seek  to  link  groups  horizontally.  Organizational  boundaries 
create  trust  problems  between  groups,  and  information  technology  that  links  a  group  with 
distrusted  others  may  be  rejected  as  too  risky,  or  irrelevant  (Baba  1994).  Other  elements  of 
organizational  structure  that  may  influence  receptivity  to  information  techjiology  include  if  the 
work  group  is  co-located,  and  if  they  travel  frequently  (Kiesler  and  Sproull  1991). 

Hypothesis  #9:  Basket  SPOs  would  not  be  the  most  likely  candidates  for  new  technology 
adoption,  since  their  resources  would  be  distributed  across  many  different  programs,  not 
allowing  the  concentration  required  for  major  technology  purchases  (see  also  Aldrich  1979). 
Large,  single  program  SPOs,  on  the  other  hand,  would  be  more  likely  to  have  a  concentration 
of  resources  needed  to  fund  new  technology. 

Age  of  Organization.  Stinchcombe  (1990)  discovered  that  organizations  bear  birth  marks 
from  the  era  in  which  their  structural  form  was  invented.  Likewise,  organizations  adopt  the 
technology  that  is  current  during  their  period  of  formation,  simultaneously  developing 
structures  and  cultures  that  are  compatible  with  these  technologies. 

Hypothesis  #10;  Programs  bom  more  recently  are  bom  more  likely  with  advanced  information 
technologies  already  in  place,  and  also  would  display  structures  and  cultures  that  were 
compatible  with  these  technologies. 

Occupational  Prestige.  Part  of  the  informal  organization  of  all  work  organizations  is  a  status 
and  prestige  hierarchy  among  occupations,  and  units  that  house  these  occupations  (Briody  et  al 
1995).  The  status  hierarchies  found  in  organizations  are  imported  largely  from  the  surrounding 
society  (i.e.,  the  environment),  but  once  imported  they  become  an  integral  part  of  the 
socio-technical  system.  Prestige  can  influence  the  adoption  of  new  technologies  in  a  variety  of 
ways  (Roberts  1994).  Groups  generally  will  adopt  technologies  that  enable  them  to  enhance  or 
maintain  their  prestige,  while  rejecting  technologies  that  challenge  their  privileged  position. 
Whether  the  technology  is  adopted  or  rejected  depends  on  the  group's  view  of  its  impact  on 
their  status. 

Hypothesis  #11:  Since  relatively  lower  status  groups  (e.g.,  logistics  vs  engineering)  have  the 
most  to  gain,  we  hypothesized  that  they  would  be  most  likely  to  experiment  with  new 
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technologies  that  could  enhance  their  capabilities  and  perhaps  their  stature  in  the  organization. 


Discipline  or  Function.  The  discipline  or  function  that  is  dominant  in  a  work  group  can 
influence  the  technology-related  behavior  of  individuals  in  the  group.  Discipline-based 
professions  and  occupations  have  subcultural  characteristics  rooted  in  the  historical 
development  of  the  discipline,  and  in  the  type  of  work  performed  by  members  of  the  discipline 
(e  g.,  physicians,  salespeople,  engineers;  van  Maanen  and  Barley  1984,  Rice  1993).  These 
subcultural  characteristics  also  are  imported  from  the  surrounding  society.  Each  discipline  has 
a  distinctive  relationship  with  technology  ~  the  role  and  importance  of  technology,  the  type  of 
technology  that  is  most  familiar,  and  the  way  technology  is  used  all  may  be  subcultural 
characteristics  (i.e.,  are  shaped  by  shared,  historically-grounded  patterns  of  behavior  and 
belief).  Some  disciplines  may  be  more  comfortable  with  certain  types  of  technology,  but  also 
may  have  stereotypical  views  of  other  technologies  that  serve  as  barriers  to  technology 
adoption  (e.g.,  engineers  often  believe  they  are  masters  of  technology,  but  older  engineers  may 
view  keyboards  as  symbolic  of  the  clerical  function). 

Hypothesis  #12;  Disciplines  or  occupations  that  are  most  familiar  with  CALS-like  information 
technologies,  and  which  view  the  use  of  such  technologies  as  an  appropriate  aspect  of 
occupational  behavior,  could  encourage  greater  receptivity  to  the  adoption  of  such 
technologies  among  their  members. 

Implementation  Process.  Research  has  shown  that  organizations  with  a  deliberate 
implementation  process  (typically  including  acquisition  planning,  user  involvement,  leadership 
roles,  training,  deployment  schedules,  and  user  support)  have  far  greater  success  at 
implementing  new  systems  than  those  that  simply  load  software  and  expect  it  to  be  used 
(Toraatzky  and  Fleischer  1990).  Our  review  of  DoD  literature  found  little  guidance  on 
implementation  processes,  suggesting  that  (particularly  given  SPO  autonomy)  there  would  be 
great  variation  among  SPOs. 

Hypothesis  #13:  Organizations  that  create  and  execute  an  implementation  plan,  particularly 
when  the  plan  includes  user  roles,  training,  and  support,  will  have  greater  success  than  those 
organizations  that  do  not. 


In  sum,  our  conceptual  architecture  embraced  thirteen  environmental  and  sociotechnical  systems 
variables.  These  were  operationalized  through  an  interview  protocol,  and  their  association  with  levels 
of  information  technology  usage,  attitudes  and  policies,  were  examined  in  the  different  field  sites. 
Although  we  derived  a  hypothesis  for  each  of  these  variables,  the  hypothesis  had  more  the  status  of  a 
"first  cut"  at  explaining  how  the  variable  operated.  This  permitted  us  to  see  the  effects  of  the  variables 
in  some  new  and  unexpected  ways. 
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Environmental  variables 


Volume  of  paperwork 
Resource  abundance 
Turbulence 
Supplier  Environment 
Air  Force  culture 


Variables  on  the  sociotechnical  interface 
lU  Legacy  systems 
£\  Organizational  characteristics 
^  Discipline  or  function 


Figure  4  -  Conceptual  Map 
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4.  Approach 


The  approach  chosen  for  developing  the  FRAMEAVORK  tool  was  an  integration  of  cultural 
anthropology,  management  theory,  and  engineering  development.  Integrating  these  standalone 
disciplines,  for  purposes  of  both  conducting  the  study  and  developing  the  readiness  assessment  tool, 
required  some  adjustment  of  their  conceptual  boundaries  and  a  willingness  to  synthesize  previously 
unrelated  fields. 


4.1  Methodology 


Our  core  focus  was  on  the  human  and  cultural  factors  in  implementation.  Given  this,  we  were 
immediately  feced  with  the  difficulty  that  there  is  no  standardization  for  these  factors  or  issues  in  the 
literature,  industry,  or  the  military.  Unlike  performance  issues  in  information  systems,  where  there  are 
standard  measurements  and  benchmarks  (mips,  baud  rates,  gigabytes  of  storage),  there  is  no  canonical 
statement  of  ^  barriers  or  readiness  fectors.  Where  there  is  consensus,  such  as  on  the  need  for 
champions  or  process  alignment,  there  are  no  standardized  measurements. 

To  understand  these  variables  and  develop  measurements  for  the  AFMC  context,  we  chose  a  field 
ethnographic  approach,  examining  cultural  patterns  inside  1 1  SPOs  and  ALC  sites.  In  ethnographic 
research,  the  observer  is  searching  for  recurrent  patterns  of  behavior  and  belief  The  ethnographer  ~ 
usually  a  solo  practitioner  --  approaches  the  site  naively,  and  immerses  himself  or  herself  in  the  setting. 
In  contrast  to  a  laboratory  or  survey  approach,  the  ethnographer  is  observing  behavior  in  a  naturalistic 
setting;  behaviors  or  beliefs  that  might  be  suppressed  or  hidden  in  laboratory  or  survey  studies,  are 
revealed  in  the  naturalistic  ethnographic  setting.  Some  of  the  observations  that  this  yielded  are 
described  below. 
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Figure  5  —  Research  and  Tool  Development  Approach 


The  goal  in  ethnographic  research  is  the  discovery  and  validation  of  these  patterns  of  belief  and 
behavior.  Given  the  nature  of  the  sample  and  the  broad  focus  of  the  inquiry,  the  ethnographer  is  less 
concerned  with  statistical  reliability  or  confidence  intervals,  and  more  concerned  with  meaningful 
patterns.  There  are  advantages  and  disadvantages  to  this  approach;  the  disadvantages  are  the  dangers 
of  insufficient  depth  in  observation,  yielding  superficial  patterns;  the  advantages  are  that  when  done 
well,  with  sufficient  depth  and  discipline,  the  ethnographic  report  that  can  communicate  the 
multidimensionality  of  a  given  situation. 

As  cm  example  of  type  of  findings  that  ethnography  yields,  one  might  consider  the  status 
differences  between  functions  in  AFMC.  Specifically,  the  logistics  function  represents  what 
Greenberg  would  characterize  as  a  "marked"  category  (Greenberg  1966).  Ahead  of  every 
other  function  including  program  control  ("bean  counters"),  the  "loggies"  are  characterized 
by  several  colorful  terms,  such  as  "wrench  turners".  Efforts  to  counteract  this  (in  the  spirit  of 
IWSM)  have  invented  awkward  terms  such  as  "maintainers"  for  the  "loggies".  One  makes 
several  observations:  (a)  the  "loggies"  being  at  the  end  of  the  developmental  food  chain, 
have  to  correct  all  the  problems  created  by  other  functions  in  order  to  make  the  aircraft  fly; 
(b)  they  work  in  what  we  sometimes  physically  disagreeable  conditions;  and  (c)  the 
chwacterization  "wrench  turners"  implies  that  they  work  only  with  their  hands,  not  their 
heads  (even  though  this  is  quite  false).  In  other  words,  we  accximidated  a  number  of 
qualitative  observations  that  form  a  consistent  pattern  of  the  logisticians  as  having  a 
subordinate  status. 
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We  chose  an  ethnographic  field  approach  because  no  other  approach  can  anticipate  all  of  the  situational 
contingencies  of  a  given  hurtian  setting  --  contingencies  that  may  be  critical  yet  overlooked  by  more 
structured  methods.  For  exknple,  a  serendipitous  finding  of  the  study  was  the  importance  of  diversity 
in  the  computer  support  group.  As  elaborated  in  section  6,  we  concluded  that  it  was  important  for  the 
demographics  of  the  support  group  to  reflect  tho.se  of  the  group  supported.  Particularly  if  the  user 
group  is  heavily  minority,  female,  or  less  educated,  a  support  group  that  is  exclusively  white  male 
computer  specialists  wiU  have  limited  eflfectiveness.  This  is  a  finding  that  initially  was  nowhere  on  our 
schedule;  we  were  not  looking  for  this,  yet  it  became  apparent  on  the  basis  of  immersion  in  the  SPOs. 


4.2  Research  Design 


Having  chosen  the  ethnographic  approach,  an  immediate  problem  that  presented  itself  was  its 
application  to  AFMC.  To  the  best  of  our  knowledge  we  are  the  first  ethnographers  to  conduct  a  study 
of  the  Air  Force  Materiel  Command.  The  traditional  ethnographic  setting  has  been  the  village,  a  small 
group  of  100  to  500  persons  living  and  working  in  reasonable  proximity.  AFMC  by  contrast  is  several 
orders  of  magnitude  larger  in  scope  and  scale:  AFMC  has  more  than  1 10,000  employees,  consisting  of 
35  thousand  military  and  75  thousand  civilian;  Wright-Pattemson  AFB,  the  home  of  AFMC  (as  well  as 
numerous  other  USAF  components)  has  28  thousand  military  and  civilian  workers  spread  out  over  13 
square  miles. 


Unit  of  Study  and  Sample  Definition 

We  decided  that  our  basic  unit  of  study  would  by  the  Systems  Program  OflSce,  or  SPO.  A  finding  late 
in  the  study  was  that  there  was  less  than  universal  understanding  in  the  Air  Force  regarding  what  is  a 
SPO.  For  practical  purposes  we  defined  a  SPO  as  a  two-letter  directorate  within  the  Aeronautical 
Systems  Center  (ASC),  the  Electronic  Systems  Center  (ESC),  or  the  Missile  Systems  Center  (MSC). 
However,  ongoing  reorganization  fi'equently  undercut  this  decision:  the  Joint  STARS  program 
changed  its  oflBce  symbol  during  the  course  of  the  study;  the  FACTS  program  (Fasteners,  Actuators, 
Connectors,  and  Tools)  (ASC/SMGH)  was  sometimes  considered  a  separate  SPO  (rather  than 
program)  within  the  Subsystems  SPO  (ASC/SM).  The  C-17  SATAF  at  Kelly  AFB  was  fi'equently 
referred  to  as  "SPO  South".  The  IWSM  (Integrated  Weapons  System  Management)  philosophy, 
which  eliminated  the  barriers  between  SPOs  and  ALCs  regarding  system  lifecycle  responsibilities,  was 
initiated  during  the  study.  In  some  respects  our  eflForts  to  study  acquisition  and  logistics  cultures  were 
like  an  effort  to  check  the  tire  presssure  on  a  fast-moving  truck. 


Wizdom  Systems,  Inc. 


FRAMEAVORK  Final  Report,  page  24 


December  11,  1995 


Unique  Sample  Issues 


This  discussion  of  our  sample  definition  is  significant,  because  it  builds  on  and  refines  two  critical 
issues.  The  first  is  the  ongoing  turbulence  within  the  Air  force  environment.  With  the  victory  in  the 
Cold  War  and  the  downsizing  of  our  military  forces,  the  Air  Force  and  other  services  are 
experiencing  in  the  1990s  the  turbulence  that  the  corporate  world  experienced  in  the  1980s.  This 
turbulence  has  a  negative  impact  on  systems  implementation,  as  described  below.  The  difficult 
paradox  of  many  Air  Force  components  this  day  is  they  are  required  to  do  more  with  less,  even  as 
the  fast-moving  dynamics  of  “more”  and  “less”  interfere  with  the  adoption  of  tools— information 
systems— that  might  make  it  possible. 

Second,  there  is  within  AFMC  a  strong  tradition  of  SPO  autonomy.  A  strong  case  could  be  made 
that  the  more  significant  culture-bearing  groups  are  the  functional  two-letter  directorates  (ASC/EN, 
engineering;  ASC/PK,  contracting;  ASC/FM,  program  control,  etc.);  indeed,  we  heard  of  and 
observed  fascinating  cultural  traits  in  many  of  these  organizations.  However,  it  is  within  the  SPO 
that  schedules,  budgets,  supplier  capabilities,  technologies,  personalities,  and  customer  requirements 
must  all  be  integrated;  the  introduction  of  new  information  systems  is  usually  a  dependent  variable  in 
this  equation.  The  autonomy  granted  to  the  SPOs  is  a  recognition  of  the  difficulty  of  this  task.  It  is 
within  the  SPO  where  the  different  human  forces  we  observed  bearing  on  innovation  were  played 
out. 


Scope  of  investigation 

Our  study  focussed  on  the  implementation  of  seven  specific  types  of  systems  within  the  SPOS.  After 
our  preliminary  field  inquiries,  we  concluded  that  CALS  was  not  a  viable  domain  of  study  because 
of  the  low  level  of  implementation.  Further,  in  terms  of  human  (as  contrasted  to  engineering  or 
management)  issues,  there  seemed  to  be  few  issues  that  were  unique  to  CALS;  most  seemed  to  bear 
on  a  variety  of  information  systems.  However,  “information  systems”  is  a  broad  category  that  could 
potentially  include  typewriters,  mainframe  computers,  fax  machines,  5-ply  carbonless  paper  forms, 
and  everything  in  between.  We  established  three  basic  criteria  to  determine  which  information 
systems  we  would  focus  on.  The  information  systems  had  to  be  systems  that: 

1 .  Were  visible  to  the  user;  this  focussed  us  on  applications,  rather  than  networks  or 
operating  systems; 

2.  Created  new  forms  of  connectivity  and  communication  among  users;  and 

3.  Required  the  alteration  of  work  routines  and  patterns  for  their  effective 
implementation. 
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Criterion  #1  excluded  Local  Area  Networks  (LANs)  and  mainframes,  although  applications  running  on 
these  might  be  included  (we  found,  in  both  user  and  support  groups,  no  clear-cut  distinctions  among 
applications,  communications,  and  hardware  platforms;  "the  system"  or  "the  computer"  usually  referred 
to  some  integration  of  all  of  these).  Criterion  #2  excluded  standalone  applications  such  as  word 
processors  and  spreadsheets.  Criterion  #3  excluded  telephones  and  fax  machines;  although  one  could 
suggest  that  these  create  new  forms  of  connectivity  and  communication  (especially  IVR,  or  Interactive 
Voice  Response  systems),  there  is  little  evidence  that  work  routines,  particularly  in  line  organizations, 
have  been  modified  to  leverage  their  potential. 

Applying  these  tests,  we  established  seven  classes  of  systems  whose  implementation  we  would  study. 
These  are: 

1 .  E-maU;  although  a  familiar  technology  in  most  parts  of  AFMC,  we  found  that  in  some 
organizations  this  was  still  cutting  edge,  and  causing  considerable  turmoil  in  its 
implementation.  Examples  of  e-mail  systems  include  PROFS,  CC:mail,  and  All-in- 
One. 

2.  Shared  Database  Systems  are  the  largest  class  of  systems  used  in  the  SPOs  and 
ALCs.  They  are  repositories  of  information  that  may  be  accessed  by  different  users, 
often  using  different  types  of  terminals  located  in  different  places,  either  on  a  client- 
server  or  mainframe  architecture.  Examples  of  shared  databases  include:  C-PAS 
Mapper;  SPO-MIS;  SPLIC. 

3.  CAD/CAM  tools.  Computer-Aided  Design  (CAD)  tools  automate  many  of  the 
functions  of  engineering  design,  including  drafting,  analysis,  and  version  maintenance. 
Computer-Aided  Manufacturing  (CAM)  tools  create  machine  instructions  from 
engineering  information.  The  integration  of  CAD  and  CAM,  essential  for  integrated 
product  and  process  development,  is  more  frequently  wished  for  than  achieved. 

4.  Electronic  Data  Interchange  (EDI)  is  the  exchange  of  business  information, 
including  quotations,  orders,  and  payments,  between  business  partners  across 
communication  lines.  EDI  uses  a  large  number  of  electronic  forms  or  transaction  sets, 
standardized  by  the  ANSI  X.  12  or  EDIFACT  standards, 

5.  Video  Teleconferencing  applications  provide  remote  meeting  capability  by  providing 
(usually  less  than  full-motion)  video  transmission  over  telephone  lines.  Video 
teleconference  participants  can  see  each  others'  faces  and  hear  each  others'  voices;  due 
to  the  data  compression  and  transmission  rates,  motion  is  typically  jerky.  Use  of 
satellite  links  creates  propagation  delays  resulting  in  a  not-quite  synchronous  dialogue. 

6.  Document  Imaging  Systems  scan  paper  documents  into  an  electronic  file,  typically  in 
rasterized  form  with  data  compression  capability.  These  systems  have  very  large 
memory  requirements. 
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7.  Workflow  tools  automate  many  aspects  of  business  processes.  When  a  workflow  tool 
such  as  Lotus  Notes  is  configured  with  a  specific  process,  it  manages  the  process  by 
routing  documents,  maintaining  schedules,  and  storing  documents,  according  to 
process  instructions. 

By  restricting  our  investigation  to  a  specific  range  of  contexts  --  development  and  support 
environments  in  AFMC,  and  a  specific  range  of  technologies,  we  have  been  able  to  focus  on  issues  that 
will  be  meaningful  to  technology  managers  in  those  contexts. 


4.3  Variables  examined 


In  the  field  research,  the  seven  technologies  listed  above  were  treated  as  dependent  variables;  that  is, 
their  presence  (or  absence),  and  level  of  usage,  were  what  we  sought  to  explain.  Based  on  our 
ecological  and  sociotechnical  ^sterns  models,  we  established  12  variables  that  were  examined  at  each 
site.  These  variables,  with  supporting  detail,  are: 

A.  Type  of  Organization,  including: 

1 .  Organizational  mission:  what  kind  of  work  is  performed  in  the  organization 

2.  Organizational  design:  formal  organizational  structure 

3.  Team  structure:  what  kinds  of  teams  are  formed,  and  who  is  included  on  the  teams 

4.  Age  of  organization:  when  the  organization  was  started 

5.  Physical  environment:  the  physical  conditions  of  the  building  and  work  areas 

B.  Group  size:  how  many  people  are  part  of  each  work  group 

C.  Fragmentation,  including: 

1 .  Occupation  and  function  of  each  individual  interviewed:  what  their  tasks  include 

2.  Workprocess  used  to  perform  the  various  functions:  flow  of  work  in  the  organization 

3.  Tasksharing:  whether  specific  tasks  are  shared  or  individualized 

4.  Collocation:  whether  people  who  work  together  sit  together,  other  locations  of 
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organizational  members,  and  the  amount  of  travel  personnel  routinely  expect 
5.  Internal  relations:  social  relations  inside  the  office  and  outside  it 

D.  Turbulence,  including: 

1 .  Organizational  turbulence:  changes  being  experienced  by  the  organization,  including 

organizational  structure'fnd  mission  changes,  changes  in  the  number  of 
personnel,  changes  in  m^agement,  task  changes,  and  changes  in  location 

2.  Perceived  risk  of  organizational  turbulence:  how  much  risk  was  associated  with 

organizational  changes  taking  pla9e 

E.  Implementation  Process,  including: 

1.  Technological  change  process:  how  new  office  information  and  communication 

technology  is  iniplemented 

2.  History  of  technological  change:  past  implementation  attempts,  successes  and  failures 

3.  Technology  change  plans:  plans  for  new  technology  implementation 

4.  Champions  and  anti-champions:  the  presence  and  activity  ofpeople  who  champion 

new  technology  implementation  and  usage  and  those  who  actively  resist  it 

5.  Computer  support:  the  size  and  diversity  of  the  computer  support  group,  as  well  as 
support  activities 

6.  Training:  amount  of  information  and  communication  technology  training  taken  by  each 

individual,  and  the  amount  and  availability  of  training  offered  by  the  organization 

F.  Technology  Attitudes,  including: 

1 .  Personal  attitudes:  positive  and  negative  attitudes  toward  information  and 
communication  tectaiology  expressed  by  each  individual  and  group  interviewed 

2.  Cultural  attitudes:  interviewees'  perceptions  of  other  people's  attitudes  about  technology, 

the  perception  of  the  group's  overall  attitudes 

3.  View  of  management:  interviewees'  perceptions  of  management's  attitudes  toward 

technology,  and  their  perceptions  of  management's  abilities  and  effectiveness 

G.  Prestige:  organizational  prestige  in  Air  Force  as  well  as  differences  in  functional  prestige 
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H.  Stage;  the  stage  of  the  product  development  c^cle  in  which  the  group  works 


I.  Security:  amount  of  secure  data  dealt  with  by  the  organization,  and  security  measures  taken  by  the 

organization  to  protect  data 

J.  Funding:  whether  the  budget  is  stable,  or  going  up  or  down;  who  is  responsible  for 

spending  budgeted  dollars 

K.  Paper  volume:  amount  of  paper  required  for  completion  of  the  work 

L.  External  relations:  relations  with  vendors  and  contractors,  and  the  technological  push  or  pull 
involved  in  the  relationship 

For  each  organization,  a  list  of  the  existing  independent  variables  was  developed  and  the  relationship 
between  the  independent  and  dependent  variables  was  examined.  Then  the  individual  organizational 
results  were  compared  with  one  another  to  develop  an  estimation  of  the  overall  effects  of  the 
environmental  and  cultural  variables  on  the  implementation  and  usage  of  office  information  and 
communication  technologies. 


4.4  Field  Techniques 


Our  field  techniques  consisted  of  both  observation  and  interviewing;  as  we  gained  greater  femiliarity 
with  SPO  routines,  interviewing  occupied  a  greater  proportion  of  our  efforts.  In  the  field  work,  a 
fieldworker  would  spend  anywhere  fi’om  three  days  to  three  weeks  at  the  site,  interviewing  up  to  35 
persons,  and  observing  behavior  firsthand.  Observation  was  important,  because  occasionally 
interviewees  would  report  on  the  existence  of  a  particular  system,  such  as  a  document  imaging  system 
or  a  CAD  seat;  yet  close  visual  inspection  would  find  several  layers  of  dust  on  the  keyboard,  and 
followup  questioning  would  reveal  that  the  system  had  not  been  used  in  the  three  years  since  the  initial 
champion  left  the  SPO.  From  a  human  point  of  view,  a  system  that  is  not  used  is  a  nullity. 

The  use  of  visual  observation  was  additionally  important  in  developing  several  of  our  cultural  findings. 
For  example,  the  casual  observation  of  interaction  during  a  pizza  party  is  informative  of  the  cohesion  of 
work  groups  and  fimctions,  or  the  indications  of  division  self-image  and  morale  in  the  decorations  of 
office  walls  and  cubicles. 

Our  interview  protocol  is  given  in  Appendix  A.  This  protocol  would  be  administered  to  anywhere 
from  fifteen  to  35  respondents.  The  interviewer  would  take  extensive  notes,  as  close  to  verbatim  as 
possible.  The  interviews  themselves  were  updated  nightly,  and  supplemental  material  was  added  to 
them  as  it  was  received.  Organizational  documents  such  as  organization  charts  and  sections  of 
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briefings  were  also  collected.  After  the  interviewing  process  was  completed,  the  interviews  were 
transcribed  and  the  interviews  were  reread.  Analysis  took  the  form  of  content  analysis  -  a  process  of 
coding,  cross-coding,  summarizing,  and  reaching  conclusions  about  the  relationships  between 
variables.  Each  phrase  or  phrase  group  was  coded  with  an  identifier  that  linked  this  part  of  the 
interview  to  one  of  the  variables  being  investigated  in  the  project.  After  coding,  for  each  interview  the 
phrases  were  grouped  by  code.  Summaries  of  the  cross-coded  interviews  were  made  and  ftie 
summaries  were  compared  and  contrasted,  to  come  to  overall  conclusions  about  the  relationships 
between  communication  and  information  technology  usage,  and  the  multiple  organizational  and 
cultural  variables  in  the  data.  These  conclusions  were  condensed,  and  the  findings  were  reported  back 
to  each  Air  Force  organization  in  a  one  to  three  hour  out-briefing.  This  gave  the  organization  a  chance 
to  review  and  comment  on  the  findings  before  the  final  cross-organizational  analysis.  This  process  was 
completed  for  all  Air  Force  organizations  studied., 

The  interviews  were  conducted  according  to  semi-structured  protocols:  one  set  of  questions  for  the 
front  office,  and  another  set  of  questions  for  the  rest  of  the  people  in  the  organization.  These  protocols 
were  a  guide  for  the  interviews,  and  were  not  used  verbatim.  Probing  questions  were  always  used  to 
expand  the  amount  of  infi^rmation  collected.  The  interview  protocols  were  constantly  under  review 
during  the  project,  to  ensure  that  the  questions  that  were  asked  were  questions  that  evoked  the  data 
necessary  to  make  the  connections  between  the  variables  being  investigated.  At  the  first,  second,  and 
fourth  organizations  studied,  the  interview  protocols  were  revised  significantly  to  bring  the  questions 
more  in  line  with  the  questions  actually  being  asked.  During  the  interviewing  process  at  the  first  and 
second  organizations  a  third  protocol  was  devised,  fi^r  management  personnel  other  than  the  division 
chief  and  deputy  division  chief,  which  used  questions  from  both  previously  created  protocols.  After 
interviewing  at  the  fourth  organization,  a  new  set  of  questions  was  formulated,  specific  to  computer 
support  personnel.  Further,  the  variables  being  investigated  were  expanded  and  narrowed  as  the 
interview  information  warranted;  some  of  the  variables  that  were  considered  significant  in  the  beginning 
of  the  project  were  not  considered  significant  by  the  end  of  the  project,  and  new  variables,  ones  not 
previously  considered,  were  added,  and  this  required  constant  minor  revisions  of  the  questions  in  the 
interview  guides. 


Coding  and  Analysis 

The  typewritten  notes  were  then  coded.  The  initial  set  of  codes  was  based  on  our  Phase  I  re^ts, 
being  primarily  an  amplification  of  the  ecological  model.  As  new  themes  emerged  from  the  interviews, 
new  codes  were  added  and  re-applied  to  earlier  interviews.  The  result  was  a  series  of  NNN  summary 
interview  transcripts,  with  segments  of  text  —  phrases,  sentences,  paragraphs  —  tagged  for  different 
themes. 

Once  the  interviews  were  coded,  they  were  then  cross-coded  --  that  is,  aU  text  segments  having  the 
same  code  were  tabulated  together.  For  example,  at  one  site,  the  following  statements  were  made  by 
different  individuals  regarding  computer  support  and  training: 
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Statements  coded  for  Computer  Suj^rt,  Training,  and  Champions,  Site  *** 

"If  you  had  somebody  to  help  you  when  you’re  on-line,  then  it  would  be  good.  The 
computer  kicks  you  off-line  after  5  minutes  of  not  hitting  a  key,  so  it's  hard  to 
resolve  problems.  We  have  several  people  who  are  knowledgeable  about  different 
Q^ms.  2  people  I  rely  on  for  help  are  in  another  Ixulding." 

no  computer  training  in  college  of  AF  except  a  2  hour  familiarization  course 
which  was  basically  how  to  turn  it  off  and  on. 

"Nobo<fy  uses  the  computer  to  its  fullest  capability.  You  live  and  learn  on  it. 
WeVe  never  had  any  class  on  computer  usage  so  we  cant  use  it  to  the  fullest 
capacity. 

Also  training  is  important  -  a  lot  of  pet^le  are  narrow  min<fcd  and  dont  see  the 
need  for  training,  but  if  you  keq)  others  from  learning  the  system,  then  what 
hai^pens  if  the  person  that  knows  how  to  do  it  gets  sick?" 

Bj  jgg  gjjg.g  saying  we  need  this,  we  n^eed  that,  and  she  tries 

hard  to  get  ft.  She's  alw^  dashing  around,  1  don't  know  where  she  gets  her 
energy." 

The  computer  experts  in  ***  make  changes  and  then  give  training  for  them." 

He  has  had  approxinmtely  5  courses  on  the  computer,  about  3  since  he  came  here 
to  the  All  these  courses  were  spedfic  to  his  job.  "You  have  to  go  to  these, 
they  have  a  list  of  courses  you  need  for  your  job  -  we  have  to  be  trained  for  level 
one  acquisitions." 

When  we  have  a  change  in  technology  they  train  you  for  it 

"No  one  I  know  is  a  champion  for  change.  We’re  all  too  caught  up  in  our  work. 
New  technolos^  comes  in  when  somdxxfy  at  the  Pentagon  decides  it's  time.  Who 
knows  who  it  is." 


In  comparison  with  the  other  SPOs  studied,  one  observes  in  this  SPO  a  relatively  low  level  of  training 
and  support. 

From  the  field  investigation  we  derived  a  set  of  management  issues.  These  form  an  array  of 
operational  choices  that  the  SPO  director,  division  chief,  or  branch  manager  can  have  some  effect  on, 
and  thereby  influence  the  readiness  of  his  or  her  organization  to  adopt  new  systems.  As  we  were 
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midway  through  the  fieldwork,  we  realized  that  we  had  a  choice  of  pursuing  findings  that  would  build 
an  elegant  theoretical  model,  or  findings  that  would  be  usefiil  to  managers: 


academic  issues 


management  issues 


Figure  6  ~  Balance  of  issues 

As  contrasted  to  these  attributes  of  the  ecological  model,  the  management  issues  are  opportunities  for 
management  intervention  that  satisfy  three  criteria: 

•  They  are  within  the  manager's  scope. 

•  They  represent  strategic  or  tactical  choices,  which  must  be  prioritized. 

•  They  will  have  a  positive  impact  on  readiness  or  capability  to  implement  new  systems. 

One  might  visualize  the  management  issues  in  terms  of  a  three-schema  architecture,  in  which  the  users 
behavior  and  experience  on  the  one  hand,  and  the  management  issues  on  the  other,  are  both  external 
views  of  a  common,  neutral  cultural  reality. 


Wizdom  Systems,  Inc. 


FRAME/WORK  Final  Report,  page  32 


December  11,  1995 


FRAME/WORK  3 -schema  architecture 


User  view:  keyboards 
and  screens 


Neutral  view: 
roles,  statuses, 
cultural  values, 
barriers 


Attitudes 


Security 


$ 

Funding 

Implementation 


Org.  Structure 


Management  view: 
structure,  process, 
funding ... 


Figure  7  -  Three  schema  view 


Our  challenge  was  to  create  a  finite  set  of  management  issues  out  of  our  research  findings.  We  began 
with  a  list  of  more  than  25  coded  variables  that  emerged  in  the  fieldwork.  These  were  then  reduced  to 
19  that  had  a  first-order  relationship  to  implementation  success;  this  list  was  then  reduced  to  16  issues 
that  were  accessible  to  management  intervention.  For  example,  a  variable,  "program  stage"  had  a 
relationship  to  a  work  group's  interest  in  new  systems,  with  the  more  downstream  disciplines 
(particularly  logistics)  being  more  favorably  disposed  to  adopting  new  systems.  However,  a  program 
manager  or  SPO  director  has  little  influence  over  the  current  stage  of  a  program.  Hence  the  finding 
that  program  stage  bears  on  attitudes  toward  new  systems,  while  interesting  philosophically,  has  little 
practical  value. 

The  management  issues  that  we  ended  up  with  are: 

•  Technology  implementation  process 

•  Training 

•  Cultural  assumptions  (attitudes)  about  computing 

•  User  support  and  diversity  of  support  group 

•  Levels  of  usage  of  computer  systems 

•  Previous  experience  with  computing 

•  Technology  champions  and  anti-champions 

•  Communication  among  co-workers 
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•  Funding 

•  Job  design 

•  Computer  literacy 

•  Computing  and  telecommunications  policy 

•  Security 

•  Organizational  barriers  and  boundaries 

•  Relationships  with  contractors 

•  Physical  access  to  end-user  devices 

After  the  set  of  management  issues  was  created,  we  created  recommendations  for  each  issue.  These 
recommendations  were  drawn  from  industiy  experience.  Air  Force  experience,  literature  reviews,  and  a 
technical  interchange  meeting  held  at  Armstrong  Laboratory. 

After  the  issues  and  recommendations  were  codified,  they  were  reviewed  by  Col.  Edward  C.  Hopkins, 
USAF  (ret.),  who  made  several  valuable  suggestions  regarding  them. 


4.5  Development  of  the  Readiness  Assessment  Tool 


Parallel  with  the  development  of  the  management  issues  was  the  development  of  the  assessment  tool. 
This  tool  originally  consisted  of  more  than  300  questions  asked  of  top  management,  middle 
management,  users,  and  MIS  specialists.  These  questions  evolved  from  the  interview  protocols  that 
we  used  in  our  fieldwork.  As  shown  in  Appendix  A,  the  interview  protocol  made  extensive  use  of 
open-ended  questions;  the  requirement  for  a  forced-response  foimat  necessitated  a  more  elaborate  set 
of  questions  when  we  built  this  protocol  into  the  tool.  Following  the  beta  test,  the  questionnaire  was 
revised,  and  one  level  of  management  was  eliminated.  This  revision  of  the  assessment  tool  cut  the 
time  required  for  the  assessment  significantly. 

A  critical  issue  in  tool  development  was  the  mapping  of  the  assessment  results  to  the  management 
issues.  We  considered,  and  rejected,  an  indexing  approach,  where  numerical  ratings  on  each  issue. 
This  approach  was  rejected  because  of  the  extensive  data  that  would  be  required  to  calibrate  the 
indices.  Instead  we  chose  to  use  a  backward-chaining  mapping  of  assessment  responses  to  issues. 
Based  on  the  fieldwork  results,  assessment  questions  were  mapped  to  the  management  issues,  and  each 
alternative  response  was  assigned  a  probabalistic  rating  for  the  pertinent  issue.  Thus,  for  example,  we 
concluded  that  if  user  attitudes  were  an  issue,  there  was  an  0.70  probability  that  on  question  3 13 1,  the 
typical  user  would  give  either  the  first  or  the  third  answer.  Altogether  over  400  pairings  of  questions 
and  issues  were  created,  with  each  issue  supported  by  at  least  six  questions. 
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3132:  Which  statement  best  describes  the  level  of  computer  literacy  of  people  in 

your  work  group? 

_ The  majority  of  people  in  our  work  group  are  computer  literate,  and  use 

computers  regularly  in  their  work. 

_ The  level  of  computer  literacy  and  receptivity  to  computers  in  our  group  is 

mixed;  some  people  are  computer  literate  and  use  computers 
regularly:  others  do  not. 

_ The  majority  of  people  in  our  work  group  are  not  computer  literate  and  do  not 

use  computers  regularly  In  their  work. 

_ In  our  work  group,  more  than  half  of  the  people  may  be  described  as  highly 

computer  literate,  with  advanced  computing  skills. 

_ A  few  people  in  our  work  group,  less  than  half,  are  highly  computer  literate 

with  advanced  computing  skills. 

_ In  our  work  group,  there  are  few  or  no  individuals  with  advanced  computing 

skills. 


An  important  issue  in  the  development  of  a  diagnostic  tool  such  as  this  is  the  choice  of  strategy  for 
anticipating  contextual  contingencies.  These  contingencies,  of  course,  are  a  function  of  how  wide  a 
range  of  contexts  one  wishes  the  tool  to  address.  There  is  an  inevitable  tradeoff  between: 

•  Context  specificity  --  to  what  extent  does  the  tool  have  context  information 
programmed  into  it,  rather  than  being  collected  at  run  time; 

•  Depth  of  analysis  --  does  the  tool  provide  a  superficial  or  insightful  diagnosis;  and 

•  Intrusiveness  of  use  -  does  the  tool  require  extensive  data  input,  or  can  it  issue  a 
diagnosis  on  the  basis  of  minimal  input. 

In  other  words,  one  can  develop  a  context-neutral  tool  that  is  minimally  intrusive,  but  it  will  return  only 
a  high-level  diagnosis;  if  we  want  an  in-depth  diagnosis,  either  the  tool  must  collect  extensive 
information,  or  else  it  must  be  tailored  for  a  specific  situation  (that  is,  it  has  situational  information 
already  built  into  it). 

This  tradeoff  is  unavoidable;  one  could  say  that  successful  development  of  diagnostic  tools  is  a  function 
of  finding  the  right  balance  among  these  three  considerations,  and  coupling  it  with  an  acceptable  user 
interface. 

The  balance  that  we  struck  with  FRAMEAVORK  is: 

Context  specificity  -  tailored  to  SPOs  and  by  extension  military  program  environments. 
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Depth  of  analysis  --  one-page  briefings,  buUet  points,  on  management  issues. 

Intrusiveness  of  use  —  approximately  4  hours  of  input  total  across  at  least  10  users. 

In  the  beta  test  the  tool  reported  the  top  five  issues  that  surfaced  as  a  result  of  the 
assessment.  Observing  reactions  to  this,  we  concluded  that  such  a  one-dimensional  result  was  at  best 
mildly  interesting,  and  hardly  compelling.  Following  the  beta  test  we  revised  the  report-out  of  the  tool, 
to  compare  the  command's  view  of  each  of  the  seventeen  issues  with  the  users'. 
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FRAMEAVORK  Report  Screen 

Figure  8  —  Report-out  screen 

In  this  screen,  the  thermometer  on  the  right  gives  an  overall  rating  of  the  organization's  eflfectiveness  in 
implementing  new  systems.  The  two-dimensional  grid  compares  the  manager’s  view  of  the  issue  (y 
axis)  with  the  users'  (x  axis).  The  issues  are  represented  by  icons,  as  shown  in  the  legend  below.  By 
clicking  on  an  icon,  the  operator  brings  up  a  window  summarizing  the  issue,  with  hypertext  links  to  the 
recommendations. 

As  a  result  of  these  post-beta  improvements,  we  achieved  our  target  of  creating  an  assessment  tool  that 
could  be  eflfectively  used  by  a  SPO  in  two  days  or  less.  Current  estimates  of  time  required  for  a 
FRAMEAVORK  version  2.0  assessment  are: 
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Setup  and  configuration 
Command  configuration 
MIS  input 

Manager  &  user  input 
Review  results 


15  minutes 
30  minutes 
30  minutes 

20  minutes  each  (minimum  10  recommended) 
30  minutes 


Summary  of  Development  Strategy 

In  sum,  the  FRAMEAVORK  development  achieved  an  integration  of  inductive,  empirical  field 
research,  conceptual  synthesis,  and  engineering  ^development.  It  achieved  this  through  successive 
refinements  of  the  model  of  SPO  culture,  the  assessment  instrument  (first  embodied  in  a  paper 
questionnaire  and  later  in  the  software  tool),  and  the  actual  FRAMEAVORK  tool: 

Our  approach  to  the  FRAMEAVORK  development  could  be  summarized  as  "making  social  science 
work  for  management."  There  is  a  strong  science  base  underlying  the  tool;  as  shown  in  section  six, 
there  are  many  interesting  findings  embedded  in  the  tool.  Careful  fieldwork  and  data  collection  went 
into  developing  these  findings.  Yet  our  goal  was  not  so  much  to  be  interesting,  as  to  provide  a  useful 
tool.  This  required  that  the  science  and  the  findings  be  packaged  in  such  a  way  as  would  be  feasible 
from  an  engineering  point  of  view  and  useful  from  a  management  point  of  view.  We  submit  that  the 
resulting  tool  meets  these  tests. 

The  approach  we  created  strikes  a  balance  between  an  academically-thorough  description  of 
technology  implementation  concepts,  and  an  adaptation  of  these  concepts  to  the  pragmatic  concerns  of 
the  SPO  director  or  division  head.  The  basic  components  of  the  tool  —  the  assessment  instrument,  the 
expert  system,  and  the  issues  report  --  are  all  derived  from  or  grounded  in  field  research  and  the  social 
science  literature  of  technology  implementation. 
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5.  Research  and  Development  Activities 


Our  research  activities  in  the  FRAMEAVORK  tool  development  embraced  parallel  efforts  to  refine  a 
model  of  SPO  culture,  develop  rapid  assessment  instruments  for  assessing  the  culture,  and  embodying 
this  instrumentation  into  a  software  tool.  Significant  attention  was  given  to  assuring  that  the  tool 
would  be  optimally  useful  to  AFMC  managers. 


5.1  Project  Overview 


Phase  n  of  the  FRAMEAVORK  research  was  conducted  fi'om  October  15,  1993  to  June  30,  1995. 
The  project  team  consisted  of  the  following  personnel: 


Dr.  Allen  Batteau 

Dr.  Marietta  Baba 

Ms.  Crysta  Metcalf 

Ms.  Kathy  Fell 

Mr.  Francisco  Pulgar- Vidal 

Mr.  John  Conway 

Dr  .  Zhen  Gang  Li 


Director  of  Research,  Wizdom  Systems,  Inc. 
Professor  of  Anthropology,  Wayne  State  University 
Graduate  student  in  Anthropology,  Wayne  State  U. 
Graduate  student  in  Anthropology,  Wayne  State  U. 
Project  Engineer,  Wizdom  Systems,  Inc. 

Senior  software  engineer,  Wizdom  Systems,  Inc. 
Software  engineer,  Wizdom  Systems,  Inc. 


Contract  support  for  reviewing  project  materials  was  additionally  provided  by  Col.  Edward  C.  Hopkins 
(USAF,  ret.),  and  Ms.  Julia  Gluesing,  a  graduate  student  in  Anthropology  at  Wayne  State,  and  a 
member  of  the  Phase  I  team. 
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Figure  9  —  Focus  of  effort 
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The  dominant  activities  in  the  project,  as  illustrated  in  the  pie  chart  above,  were  extending  and 
validating  the  ecological  model,  developing  the  software  instrument,  and  engineering  development  of 
the  FRAME/WORK  tool.  Other  activities  included  identifying  successful  implementation  strategies, 
identification  of  USAF  applications,  and  the  beta  test  of  the  tool, 

Two  events  in  the  third  and  fourth  quarter  of  1994  events  created  unavoidable  schedule  delays.  The 
first  of  these  was  some  integration  problems  in  the  software  development.  This  pushed  beta 
deployment,  which  had  originally  been  scheduled  for  August  1,  1994,  back  to  early  January  1995.  A 
successful  beta  launch  was  achieved  in  January  1995  at  two  ESC  directorates. 

The  second  problem  was  the  acceptance  by  the  Principal  Investigator  of  a  professorship  at  Wayne 
State  University,  beginning  in  January  1995.  This  resulted  in  stretching  out  the  beta  deployment,  and 
delaying  the  final  software  modifications.  Neither  of  these  events  is  judged  to  have  had  an  adverse 
effect  on  the  overall  quality  of  the  final  product. 

Significant  attention  was  given  to  briefing  the  project  to  field  sites,  beta  sites,  and  other  groups.  Figure 
10  on  the  next  page  lists  all  of  the  briefings  conducted  during  the  course  of  Phase  H. 
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Date  Location 

Component 

Key  contact 

Type  of  briefing 

Result 

12/6/93  Wright-Patterson 

ASC/RE 

Col.  Bednarz 

Outbriefing 

Reported  on  findings  at  RE 

12/7/93  Kelly  AFB 

ASC/LAA 

Mr.  Martinez 

Kick  off  field  research 

Fieldwork  initiated 

12/8/93  Eglin  AFB 

ASC/VL 

Col.  Dickson 

Establish  new  field  site 

1/14/94  Wright-Patterson 

AL/HRGA 

Captain  Smith 

Phase  n  Kickoff 

1/25/94  Hanscom  AFB 

ESC/AV 

Matt  Mleziva 

Request  for  fieldwork 

Fieldwork  set  up 

2/3/94  Wright-Patterson 

ASC/YP 

David  Gentry 

Request  for  fieldwork 

Fieldwork  set  up] 

2/22/94  Wright-Patterson 

ASC/SMGH 

Col.  Saliba 

Request  for  fieldwork 

2/23/94  Wright-Patterson 

ASC/RE 

Col.  Bednarz 

Summary  outbriefing 

Combined  RE  &  Det-8 

3/1/94  Kelly  AFB 

ASC/SA 

Ray  Ranzo 

Request  for  fieldwork 

3/1/94  Kelly  AFB 

ASC/LAA 

Mr.  Martinez 

Preliminary  outbriefing 

4/2 1  /94  Hanscom  AFB 

ESC/MS 

Joseph  Mardo 

Request  for  fieldwork 

5/11/94  Wright-Patterson 

Armstrong  Lab 

Capt.  Bob  Smith 

Quarterly  Review 

5/11/94  Wright-Patterson 

ASC/CY 

Keith  Pickleheimer 

Outbriefing 

5/11/94  Wright-Patterson 

ASC/VL 

Col.  Dickson 

Assessment  briefing 

LANTIRN  set  up 

5/13/94  Wright-Patterson 

ASC/VL 

Lt.  Col.  Zlotkowski 

LANTIRN  request 

9/13/94  Wright-Patterson 

Armstrong  Lab 

Capt.  Bob  Smith 

Quarterly  Review 

9/27/94  Kelly  AFB 

ASC/SA 

Sam  Idrogo 

Request  for  fieldwork 

12/6/94  Warner  Robins  ALC 

Det-8 

Col.  Cole 

outbrief  of  Det-8 

1/4/95  Hanscom  AFB 

ESC/JS 

Major  Carol  Jones 

Joint  Stars  outbriefing 

1/4/95  Hanscom  AFB 

ESC/AV 

Capt.  Wituzynski 

Comm&  Control  outbrief 

1/19/95  Wright-Patterson 

Armstrong  Lab 

Capt.  Bob  Smith 

Quarterly  Review 

1/19/95  Wright  Patterson 

ASC/CY 

Dwight  Early 

discuss  beta  site 

1/19/95  Wright-Patterson 

FACTS 

Col  Saliba 

outbrief 

1/26/95  Eglin  AFB 

ASC/YH 

Col.  Sullivan 

outbrief  conv  muni 

Figure  10  ~  FRAMEAVORK  briefings 
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5.2  Extending  and  Validating  Model 
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Figure  1 1  -  Extending  and  Validating  the  Model 

The  original  findings  of  Phase  I  were  based  on  a  single  SPO  within  ASC.  This  case  study  provided  a 
good  foundation  for  further  model-building  in  AFMC.  Building  on  this,  a  lead  task  in  the  research  was 
to  find  a  sufficient  number  of  SPOs  and  ALC  components  to  extend  our  basic  model  of  cultural 
impacts  on  implementation.  To  do  this,  we  undertook,  firom  March  1993  to  January  1995,  fieldwork 
in  1 1  different  AFMC  organizations: 

Recon  System  Program  Office  (ASC/RE) 

C17  System  Program  Office 
Det-8  of  Recon 

C-17  Site  Activation  Task  Force  (ASC/CY-LAA) 

LANTIRN  System  Program  Office 
Contractor 

F16  System  Program  Office 
FACTS  Program 

Joint  STARS  Program  Office  (ESC/JS) 

Command  and  Control  Program  Office  (ESC/AV) 

Conventiwial  Munitions  Program  Office  (ASC/YH) 

These  research  sites  and  our  original  fieldwork  in  the  Subsystems  SPO  (ASC/SM)  form  the  basis  for 
our  model  of  SPO  culture  and  technology  implementation. 


Wizdom  Systems,  Inc. 


FRAMEAVORK  Final  Report,  page  41 


December  1 1, 1995 


Our  efforts  to  gain  access  at  an  ALC  were  unsuccessful,  despite  two  trips  to  one  specific  ALC  to 
brief  responsible  managers.  We  did,  however,  conduct  research  in  two  SPO  components  located  at 
ALCs;  these,  together  with  interviews  with  maintainers  at  another  ALC  gave  us  insight  into  some  of 
the  critical  differences  between  SPOs  and  ALCs  and  the  general  applicability  of  our  findings. 

Two  of  these  sites  did  not  yield  usable  findings:  in  one,  the  commander  who  had  given  us 
permission  to  conduct  the  study  was  re-assigned  just  before  we  began  the  fieldwork;  Interviews  with 
personnel  in  this  organization  turned  out  to  be  perfunctory  and  superficial.  In  another  organization, 
a  major  suspense  interfered  with  the  fieldwork,  also  resulting  in  unsatisfactory  interviews. 


Site  recruitment 

Field  sites  were  recruited  in  a  variety  of  ways.  Four  of  the  sites  came  through  personal  referral  from 
our  initial  contact  at  the  Subsystem  SPO.  One  site  asked  us  to  come  in  and  conduct  an  assessment. 
Five  others  were  the  result  of  a  letter  sent  to  all  SPO  directors  in  ASC  and  ESC  requesting  a  research 
opportunity.  In  every  case,  after  the  initial  contact  we  presented  a  briefing  to  the  SPO  director  or  his 
designate.  The  briefing  included  a  request  for  the  research  opportunity.  Typically  at  the  end  of 
these  briefings  a  point  of  contact  was  designated,  and  the  researcher  arranged  scheduling  with  the 
POC. 


Interviewing  and  SPO  observation 

Fieldwork  for  the  study  involved  interviewing  138  people  individually,  and  74  in  group  interviews. 
Many  of  these  people  were  conferred  with  more  than  once  in  order  to  validate  the  findings  from  the 
interviews.  The  interviews  lasted  from  one  to  two  hours  on  average.  Interviewees  were  asked  to 
speak  on  a  number  of  topics  related  to  their  use  of  information  and  communication  technologies, 
other  people’s  use  of  these  technologies,  and  their  organizational  culture.  Except  where  precluded 
by  security  considerations,  the  interviewer  was  given  tours  of  the  organizations,  and  work 
relationships  and  the  physical  environment  were  observed  before,  between,  and  after  the  interviews. 

A  number  of  observations  were  made  in  each  SPO.  In  Appendix  E  we  report  on  our  observations 
and  findings  for  each  SPO.  These  are  the  observations: 

•  Illustrate  the  qualitative  nature  of  this  research,  and  the  organization  of  visual  data 
and  unstructured  observation  to  elucidate  cultural  patterns 

•  Document  the  enormous  variability  of  local  cultures  within  AFMC. 

Such  issues  as  the  physical  layout  of  workspaces,  a  pizza  party,  or  the  decorations  on  an  office  wall, 
are  important  indicators  of  work  processes,  social  relationships,  and  organizational  morale. 
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The  fieldwork  supporting  both  the  FRAMEAVORK  tool  and  the  findings  of  this  report  was  unusually 
thorough.  The  combination  of  structured  interviewing  and  unstructured  observation  yielded  a  rich  set 
of  findings,  which  are  presented  in  section  6. 


5.3  Identify  Successful  Strategies 
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Figure  12  —  Identifying  successful  strategies 

Four  initiatives  were  undertaken  to  identify  implementation  strategies  for  dealing  with  human  issues  in 
^ems  implementation; 

1 .  Contacting  CALS  shared  resources  centers  and  CALS  vendors 

2.  Literature  review 

3 .  Industry  e)q)erience  of  the  team 

4.  Technical  interchange  meeting  of  ASC  MIS  personnel 

In  January,  1994  the  PI  contacted  the  CALS  Shared  Resources  Centers  (CSRCs)  to  elicit  any  success 
stories  they  would  have  regarding  CALS  implementation.  Although,  as  explained  in  the  previous 
section,  our  focus  was  largo’  than  CALS,  we  judged  that  CALS  was  a  useful  heading  for  identifying 
the  class  of  work-process-changing  technology  we  were  interested  in.  Three  of  the  CSRCs  responded, 
and  dght  leads  were  developed  for  success  stories.  These  woe  pursued,  and  through  interviews 
developed  into  narrative  form.  The  success  stories  have  been  embodied  in  the  tool  linked  to  the 
recommendations. 
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A  second  approach  to  successfiil  strategies  came  from  a  review  of  the  literature  on  the  implementation 
of  information  systems.  Some  of  the  leading  sources  supporting  the  recommendations  in  the  tool 
include  Adler  1990,  Baba  1995,  Handy  1995,  Majchrzak  1992,  and  Taylor  and  Felton,  1993. 

Related  to  this  was  a  third  approach,  the  use  of  industry  experience  with  systems  implementation.  The 
senior  members  of  the  team  (Dr.  Batteau  &  Dr.  Baba)  have  both  had  several  years  experience  in 
consulting  on  human  issues  in  implementation,  and  have  from  time  to  time  prepared  briefings  and 
reports  on  human  and  cultural  factors.  This  experience  and  material  was  available  to  the  project  and 
was  used  where  judged  appropriate  by  the  PI. 

As  a  final  approach,  on  July  20,  1994  the  Armstrong  Laboratory  (in  its  electronic  meeting  fecility) 
hosted  an  informal  information  exchange  to  which  were  invited  all  the  MIS  managers  from  ASC 
program  oflSces.  .^)proximately  12  individuals  attended  this  event,  and  using  the  electronic  meeting 
facility  produced  a  20-page  transcript  discussing  examples  of  successful  implementations,  barriers  to 
implementation,  and  the  critical  success  factors  in  implementation.  These  examples  are  discussed  in 
section  6. 


5.4  Create  Assessment  Instrument 


Creating  an  assessment  instrument  was  perhaps  the  most  challenging  part  of  this  project.  This  stems 
from  the  fact  that  the  variables  we  wanted  to  assess  —  user  and  organizational  readiness  for  and  bamers 
to  systems  implementation  -  are  far  from  standardized.  There  is  no  canonical  statement  of  ^  barriers 
or  readiness  fectors  in  the  literature.  There  is  consensus  on  some  issues,  such  as  the  n^  for 
champions;  on  others,  particularly  those  that  are  specific  to  the  mihtary  (such  as  the  security  issues), 
there  is  nothing  written.  Even  when  there  is  consensus  on  an  issue,  there  are  no  standardized  measmes 
for  the  issue.  The  empirical,  inductive  approach  used  in  developing  the  FRAMEAVORK  tool  provided 
a  significant  resolution  of  this  diflScuh  issue. 
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Figure  13  --  Developing  the  assessment  instrument 

Devdoping  the  Questions 

Questions  for  the  FRAMEAVORK  tool  were  created  based  on  the  interview  protocols  used  during  the 
field  research.  Some  questions,  which  were  found  to  produce  little  predictive  data,  or  which  could  not 
be  transformed  into  closed-ended  questions,  were  eliminated.  Other  questions  had  to  be  rewritten, 
sometimes  making  one  broad  question  into  three  or  four  more  specific  questions.  It  was  ensured  that  a 
range  of  questions,  specific  to  each  significant  predicting  variable,  were  asked.  Once  this  was 
accomplished,  possible  answers  to  each  question  were  gathered  from  the  data,  shortened,  and  inserted 
as  answer  choices.  This  process  of  gathaing  possible  answer  choices  from  actual  answers  in  the 
interviews  allowed  the  answer  choices  to  reflect  variability  while  establishing  validity.  The  questions 
were  then  linked  to  issues  which  reflected  the  management  agnificance  of  the  variables. 

The  questions  were  linked  to  issues  through  Bayesian  lo^c  rules.  These  logic  rules  consisted  of 
probabilities  which  expressed  the  likelihood  of  any  given  answer  being  associated  with  the  particular 
issue  that  was  linked  to  the  question.  For  each  issue  it  was  asked:  in  how  many  organizations  was  this 
issue  important?  From  this  came  the  initial  probability  for  the  issue  to  occur  in  any  Air  Force 
organization.  Then  it  was  asked:  of  the  organizations  with  this  issue  appar^t  in  them,  what  percentage 
of  people  wCTe  likely  to  choose  this  particular  answer? 

It  was  then  asked:  in  organizations  which  did  not  display  this  issue,  what  percentage  of  people  were 
likely  to  choose  this  particular  answer? 

From  the  answers  to  these  two  questions,  based  on  the  fieldwork  experience,  came  the  Bayesian  logic 
probabilities  -  the  probability  that  any  particular  answer  would  be  common  or  uncommon  in  an 
organization  with  the  issue,  common  or  uncommon  in  an  organization  without  the  issue.  For  answer 
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sets  whose  members  were  mutually  exclusive,  the  probabilities  had  to  add  up  to  1,0.  For  answer  sets 
whose  members  were  not  mutually  exclusive,  the  probabilities  did  not  have  to  add  up  to  1.0. 


The  Liveware  Session 

A  rare  and  valuable  opportunity  to  test  some  of  the  concepts  was  afforded  by  the  Reconnaisance  SPO 
in  April  1994,  when  they  agreed  to  host  a  real-time  test  of  the  questionnaire  and  evaluation.  The  team 
traveled  to  Dayton  on  April  7,  1994,  and  met  with  ASC/RE  personnel  for  approximately  four  hours, 
asking  personnel  at  all  levels  the  different  items  on  the  questionnaire.  This  yielded  some  important 
refinements  of  wording  and  focus.  The  original  intention  of  this  session  was  to  give  the  personnel 
present  feedback  on  the  team's  conclusions  of  the  assessment;  however,  the  session  was  interrupted  by 
a  toxic  gas  leak  in  the  area,  which  forced  an  evacuation  of  the  entire  building.  Accordingly,  this  report- 
back  was  conducted  by  teleconference  a  week  later,  with  valuable  results:  SPO  personnel  found  the 
conclusions  meaningful  and  "something  to  think  about". 

Following  this  session,  further  refinements  were  made  in  the  assessment  questionnaire,  which  along 
with  an  expert  system  linking  the  assessment  to  the  management  issues,  was  developed  into  the 
FRAMEAVORKtool. 


5 . 5  Engineering  Development 


Figure  14  —  Engineering  development 

Engineering  development  of  the  FRAMEAVORK  tool  began  in  December  of  1993;  the  first  beta 
release  was  in  December  of  1994.  A  requirements  document  outlined  a  basic  set  of  requirements  in 
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terms  of  collecting  user  information  (the  assessment),  analyzing  user  input,  and  presenting  a  set  of 
results  to  management. 

Below  these  basic  requirements  were  a  number  of  design  issues  involving  tradeoffs  between  the 
intrusiveness  of  the  assessment,  the  level  of  analytic  detail,  and  the  nuancing  of  management 
recommendations.  These  issues  were  resolved  through  team  meetings  involving  both  the  field 
researchers  and  the  software  developers. 

A  more  detailed  design  description  of  the  tool  is  given  in  Appendix  C. 


5.6  Initial  Deployment 


After  the  first  working  version  of  the  FRAMEAVORK  tool  was  assembled,  copies  were  sent  to  USAF 
SPOs  that  had  volunteered  time  to  conduct  initial  testing  activities.  In  the  software  development 
lifecvcle  this  phase  is  called  “beta  testing.” 

The  initial  list  of  beta  sites  consisted  of; 

Conventional  Munitions  SPO  (Eglin  AFB) 

Command  and  Control  SPO  (Hanscom  AFB) 

Reconnaissance  SPO  (WPAFB) 

J-STARS  SPO  (Hanscom  AFB) 

F-16  SPO  (WPAFB) 

Due  to  scheduling  difficulties  and  the  reassignment  of  personnel,  full  beta  tests  were  completed  only  at 
the  first  three  of  these. 


Conventional  Munitions  SPO 

At  this  site  the  tool  was  tested  by  the  MIS  manager.  This  manager  decided  that  only  he  should  test  the 
tool,  inasmuch  as  earlier  beta  tests  of  other  tools  had  left  his  users  disappointed.  His  organization  had 
just  finished  a  compulsory  survey,  and  he  judged  the  time  not  opportune  for  any  further  interruptions  in 
the  daily  work. 

One  of  this  manager’s  most  important  observations  was  that  the  language  used  in  the  questionnaires 
was  too  generic  or  obscure,  and  that  the  MIS  person  should  have  the  ability  to  include  the  right 
wording  so  that  users  would  not  be  faced  with  confusing  questions.  For  instance,  users  who  routinely 
use  workflow  tools  without  realizing  the  nature  of  the  tools  (seeing  them  as  part  of  an  e-mail  suite), 
would  be  at  a  loss  when  asked  about  the  frequency  of  use  of  a  workflow  tool.  If  MIS  had  had  the 
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opportunity  to  replace  “workflow  tool”  in  the  questionnaire  with  the  name  of  the  application  familiar  to 
the  users,  the  responses  would  have  been  more  accurate. 

Also,  the  current  structure  of  the  tool  favors  an  order  for  data  entry  that  closely  resembles  the 
organizational  hierarchy.  Although  this  is  helpful  in  the  organization  of  the  questionnaire,,  it  presents  a 
usage  hurdle:  data  entry  at  the  user  level  needs  to  wait  until  hard-to-get  personnel  at  the  directorate 
and  division  have  entered  their  data. 

He  also  made  the  remark  that  some  of  the  answer  choices  provided  were  ambiguous  or  were  not 
uniformly  graded,  and  asked  that  the  MIS  person  be  able  to  fill  in  some  of  the  “other”  choices. 

These  recommendations  were  incorporated  into  the  subsequent  beta  version,  which  was  then  tested  at 
Hanscom  AFB. 


Command  and  Control  SPO 

The  SPO  Executive  Officer  was  the  team’s  point  of  contact  at  Command  and  Control.  He  assembled 
thirteen  users  to  test  the  assessment  tool,  including  the  SPO  director  and  the  computer  specialist.  The 
total  number  of  users  answering  the  questionnaire  was  eleven. 

The  two  most  important  observations  at  this  site  are:  First,  the  language  used  in  the  questions  and 
answer  choices  was  too  general,  or  too  “techie”  for  some  users,  with  too  little  reference  to  actual 
packages.  This  was  despite  some  modifications  made  in  response  to  the  previous  beta  test.  The  team 
was  told  to  write  at  an  easy-to-understand  level.  Second,  the  questions  needs  to  be  better  tuned  to  the 
level  they  are  targeted  for,  and  some  indication  should  be  made  regarding  fi'om  what  viewpoint  the 
respondent  should  answer  the  questions. 

Other  observations  included:  not  liking  the  answer  choices,  black  or  white  alternatives  instead  of  a 
gradation,  aggravating  navigation  commands,  and  that  the  system  should  be  on-line  because  it  is  too 
difficult  to  bring  people  to  “the”  PC. 


Reconnaissance  SPO 

The  MIS  manager  and  Mr.  Francisco  Pulgar- Vidal  assembled  fifteen  people  to  test  the  tool,  including 
the  director,  five  subdivision  managers,  and  the  computer  specialist.  The  total  number  of  users 
answering  the  questionnaire  was  eight,  including  one  anti-champion. 

At  this  beta  test  we  observed  user  sessions  as  they  happened,  rather  than  being  briefed  on  them  later. 
Most  users,  including  senior  personnel,  got  used  to  the  user  interface  during  the  fiist  few  screens, 
especially  when  told  about  keyboard  shortcuts.  Some  personnel  performing  more  than  one  assignment 
had  difficulty  knowing  which  viewpoint  to  use  to  answer  some  questions.  At  times  the  answer  choices 
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provided  would  not  satisfy  a  respondent:  he  or  she  would  have  liked  to  answer  “none  of  the  above”, 
but  there  was  no  such  possibility.  Questions  asking  for  number  of  contractors  did  not  specify  number 
of  firms  or  number  of  personnel.  Users  would  at  times  complain  that  the  questionnaire  was  too  long, 
and  would  observe  that  a  duration  of  fifteen  to  twenty  minutes  was  the  practical  limit. 


5.7  Integration  with  non-CALS  systems  development 


Two  activities  were  undertaken  to  investigate  the  integration  of  the  FRAMEAVORK  tool  with  military 
systems  development  and  implementation.  The  FRAMEAVORK  team  has  reviewed  DoD  Directives 
8020.1,  8320.1-M,  8320.1-M-l,  8320.1-M-x;  these  documents  describe  DoD  procedures  for  systems 
development  and  implementation.  Based  on  this  review,  we  have  identified  nine  areas  in  DoD 
procedures  that  would  be  supported  by  the  FRAMEAVORK  tool.  These  are  described  in  section  8. 

Additionally,  at  the  suggestion  of  Joint  Stars  personnel  (ESC/JS),  we  investigated  the  manner  in  which 
the  FRAMEAVORK  tool  could  be  integrated  with  JIMIS  (Joint  STARS  Integrated  Maintenance 
Information  System),  a  portable  maintenance  aid  scheduled  for  initial  deployment  in  November  1995. 
The  PI  was  briefed  on  JIMIS  by  a  JS  MIS  manager  and  by  contractor  personnel  at  Melbourne,  FL; 
additionally  he  was  briefed  on  IMIS  concepts  by  HRGA  personnel,  and  flight  line  issues  by  a  former 
flight  line  supervisor.  Four  critical  cultural  issues  --  history  of  implementation,  turbulence, 
organizational  barriers,  and  job  design  --  were  identified  that  could  lead  to  user  resistance  in  the  JIMIS 
deployment;  these  are  discussed  in  the  more  general  context  of  PMA  devices  in  section  7. 


5.8  Conclusion 


As  can  be  seen,  the  research  and  development  on  FRAMEAVORK  consisted  of  multiple  activities 
undertaken  in  parallel.  This  enabled  the  rapid  transition  of  basic  science  findings  (the  effects  of  SPO 
culture  and  context  on  IT  implementation)  into  a  practical  tool.  As  will  be  evident  in  subsequent 
sections,  this  science  base  has  potential  for  practical  application  well  beyond  the  FRAMEAVORK  tool. 
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6.  Research  Findings 


The  findings  from  the  FRAMEAVORK  research  have  three  purposes.  One  purpose  is  to  enlarge  our 
understanding  of  the  integration  of  sociotechnical  systems.  Members  of  the  FRAMEAVORK  team  are 
preparing  scientific  papers  for  publication  in  scholarly  journals  that  will  make  a  significant  contribution 
to  the  literature  of  sociotechnology.  The  second  purpose  is  to  provide  practical  information  to  the  Air 
Force  regarding  information  technology  management  practices.  The  third  purpose  is  to  support  the 
development  of  the  FRAMEAVORK  readiness  assessment  tool. 

We  present  here  three  types  of  findings: 

•  General  findings  of  patterns  that,  while  observed  at  the  SPO  level  of  organization,  have 
broader  implications  for  AFMC  systems  implementation  policy.  These  include  the 
levels  of  implementation,  the  perceptions  of  levels  of  implementation,  and  the  role  of 
management. 

•  The  effects  of  the  external  and  internal  SPO  environment  on  implementation;  these  are 
issues  such  as  program  definition  and  infrastructure  which  the  SPO  has  some,  but  not 
exclusive  leverage  over. 

•  Issues  that  bear  on  the  sociotechnical  integration  of  the  SPO  —  how  well  its  social 
system  meshes  with  its  technological  infi-astructure.  These  are  issues  over  which  the 
SPO  director  has  significant  leverage. 


6. 1  Findings  with  broad  implications  for  AFMC 


This  first  set  of  findings,  like  the  others,  is  derived  exclusively  fi'om  observations  inside  the  SPOs. 
These  findings,  however,  have  broad  ramifications  for  AFMC  information  technology  policy. 


Finding  #1:  Levels  of  IT  usage  in  the  SPOs  fall  short  of  DoD  paperless  objectives 

Compared  to  DoD  objectives  for  highly  automated  and  paperless  environments,  office  information  and 
communication  technology  usage  is  low  in  a  majority  of  Air  Force  organizations.  Computers  tend  to 
be  used  for  word  processing  and  E-mail  in  most  of  the  organizations,  even  the  lowest  use  ones.  Shared 
databases  and  workflow  tools  are  common  in  many  of  the  organizations  studied,  but  the  quantity  used, 
number  of  people  who  use  them,  and  the  fi’equency  of  their  use  is,  more  often  than  not,  very  limited. 
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Video  teleconferencing  is  available  in  most  organizations,  but  it  is  used,  in  most  cases,  only  by  a 
minority  of  organizational  members.  Use  of  electronic  data  interchange  and  document  imaging  is 
almost  non-existent  even  in  the  highest  usage  organizations,  and  CAD/CAM  systems  are  hardly  used  at 
all. 


Finding  #2:  Perceptions  of  IT  usage  levels  correlate  inversely  with  actual  levels 

Interestingly,  personnel  in  high  usage  organizations  may  have  a  tendency  to  consider  their 
organizations  low  usage  organizations,  while  members  of  low  usage  organizations  may  feel  they  have 
higher  usage  levels  than  they  do,  compared  to  the  rest  of  the  Air  Force.  This  is  potentially  related  to 
where  new  organizational  members  are  recruited  from.  In  some  organizations,  new  people  come  from 
organizations  that  had  less,  or  approximately  the  same  amount  of  office  technology  usage,  while  in 
other  organizations,  the  new  recruits  come  from  computer  automation  test  sites,  far  more  automated 
than  the  organization  into  which  they  are  moving.  This  history  biases  the  viewpoints  of  each  group  - 
even  in  a  high  usage  organization,  members  who  came  from  a  higher  usage  organization  will  interpret 
usage  levels  as  low,  and  in  a  low  usage  organization,  members  who  are  used  to  this  level  of  usage,  or 
come  from  even  lower  usage  organizations  will  interpret  usage  levels  as  high.  In  other  words, 
interpretation  of  computer  usage  levels  is  based  on  previous  experiences.  If  you  recruit  people  from 
low-use  organizations,  other  low  to  medium-use  organizations  will  look  like  high-use  organizations;  if 
you  recruit  from  high-use  organizations,  other  high  to  medium-use  organizations  will  tend  to  pale  in 
comparison. 


Finding  #3:  The  biggest  bottlenecks  to  CALS  implementation  are  above  the  program  office 

Very  early  in  our  study  we  discovered  that  SPO  personnel  were  more  mystified  than  enthused  by 
CALS.  "A  dreamsheet"  was  one  characterization  we  heard.  To  the  extent  that  SPO  managers  were 
familiar  with  CALS  concepts,  the  general  attitude  was  that  it  was  a  good  concept,  but  they  had  more 
critical  problems  facing  them.  Overall  there  was  little  understanding  of  the  concepts  because  they  were 
not  perceived  as  having  any  criticality. 


Finding  #4:  Aggressive  implementation  strategies  work  best 

The  rapidity  of  implementation  correlates  strongly  with  organizational  usage  levels.  The  more  people 
who  start  using  the  technology  at  the  same  time,  the  more  useful  this  information  and  communication 
technology  tends  to  be.  If  only  ten  people  in  a  two  hundred  person  single  program  office  are  hooked 
together  in  order  to  communicate  and  share  information  with  one  another,  they  will  experience  this 
technology  as  far  less  useful  than  they  would  if  they  could  communicate  with  everyone  in  the 
organization.  If  only  small  numbers  are  hooked  up  initially,  and  implementation  is  slow,  the  initial  users 
may  discontinue  their  usage  of  the  technology  and  share  their  lack  of  satisfaction  with  other 
organizational  members.  This  reduces  the  likelihood  of  successful  implementation.  In  other  words,  the 
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more  quickly  the  number  of  people  who  have  and  use  the  technology  comes  to  a  critical  mass,  the 
more  likely  the  success  of  the  implementation.  The  more  people  use  the  technology,  the  more  others 
will  be  encouraged  to  use  it,  in  a  self-peipetuating,  snowball  effect. 

This  finding  is  particularly  significant  in  light  of  the  fiequent  pressures  to  deploy  (or  at  least  announce) 
new  systems  before  they  are  technically  mature.  Rapid  deployment  of  immature  systems  is  self- 
defeating;  extended  deployment  of  immature  systems,  of  which  there  are  many  examples,  tends  also  to 
give  the  program  a  bad  reputation. 


Finding  #5:  IT  implementation  presents  a  management  challenge 

Active  management  resistance  to  office  automation  technology  can  hinder  organizational  attempts  at 
implementation  and  usage,  especially  if  the  management  anti-champion  is  in  a  position  to  prevent 
implementation  and/or  usage,  but  managerial  resistance  can  also  be  overcome  by  a  receptive  overall 
culture  and  a  computer  support  group  that  champions  new  office  technology  actively.  In  general, 
management  viewpoints  do  not  tend  to  correlate  with  organizational  usage  levels.  Management 
personnel  were  seen  as  unreceptive  in  most  organizations,  and  in  others  as  not  pushing  office 
technology  implementation  or  usage.  In  only  one  of  the  organizations  studied  was  the  management 
viewed  as  receptive.  In  many  organizations,  the  management  personnel  were  reported  as  the  group 
who  used  office  information  and  communication  technology  the  least,  even  in  the  medium  and  high 
usage  organizations.  The  relatively  slight  role  of  management  attitudes  may  be  attributable  to  the 
length  of  tenure  usually  experienced  by  personnel  in  management.  Air  Force  officers  change  locations 
fi'equently,  and  a  temporary  resistance  is  much  less  influential  on  the  organization  than  a  permanent 
one.  If  these  people  have  spent  most  of  their  military  careers  performing  their  work  manually,  there 
would  be  resistance  to  changing  something  that  has  worked  well  for  them  in  the  past,  but  they  usually 
do  not  actively  bar  their  subordinates  fi’om  using  the  technology.  They  simply  do  not  lead  by  example, 
which  is  not  as  much  of  a  hindrance  as  active  anti-championing  efforts. 

Leadership  is  an  important  aspect  in  the  implementation  of  office  information  and  communication 
technology,  but  leadership  in  this  arena  appears  to  be  more  important  if  it  comes  fi'om  computer 
support  personnel  or  fi’om  informal  champions  of  technology  change,  rather  than  from  the  SPO 
directors.  Computer  support  and  other  personnel  are,  many  times,  more  permanent  than  directors. 
When  directors  change,  policies  change  with  them,  including  at  times  office  technology  policies.  The 
changes  introduced  by  new  directors  will  often  have  limited  time  to  gain  support.  The  systems  that  are 
stable,  well-established,  and  have  widespread  organizational  support  will  continue  to  be  used.  These 
systems  will  be  systems  introduced  and  championed  by  personnel  with  long  histories  of  working  in  the 
SPO. 
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Finding  #6:  In  the  Air  Force  culture, 
engineering  and  office  automation  are  not  perceived  as  mission  critical 

The  Air  Force's  sense  of  mission  and  purpose,  which  recognizes  defense  of  the  nation  as  its  raison 
d'etre  and  views  fighter  pilots,  fighter  planes,  and  engines  as  most  vital  to  that  mission,  has  an  influence 
on  technology  change.  While  computer-aided  design  technology  does  directly  benefit  the  design  of 
fighter  engines,  the  operation  of  such  technology  is  not  the  responsiblity  of  the  Air  Force  engineers,  but 
of  outside  suppliers.  Therefore,  a  dynamic  force  for  technology  change  in  industry  (i.e.,  the  evolution 
of  CAD  technology)  is  absent  in  the  Air  Force. 


6.2  Findings  regarding  the  environment  of  SPOs  and  SPO  work  groups 

The  SPO  environment  contains  numerous  features  over  which  the  SPO  director  has  partial  leverage. 
For  some,  such  as  the  turbulence  of  the  AFMC  environment,  he  can,  at  best,  mitigate  its  elffects;  for 
others,  such  as  the  age  of  physical  infrastructure,  change  can  be  part  of  a  long-range  strategy. 


Finding  #7:  Turbulence  within  AFMC  is  a  major  inhibitor  to  IT  implementation 

At  the  time  of  the  interviewing,  the  Air  Force  as  a  whole  was  going  through  both  extensive  and 
intensive  changes.  Funding  was  decreasing,  a  mandatoiy  force  reduction  (downsizing)  was  occurring, 
bases  were  closing,  and  the  organizational  structure  as  a  whole  was  being  altered.  Acquisition  and 
sustainment  organizations,  which  had  previously  been  autonomous  and  under  separate  command 
structures,  were  being  combined  under  the  auspices  of  IWSM  (Integrated  Weapons  System 
Management)  to  produce  "cradle  to  grave"  project  management  in  the  product  development  cycle.  As 
part  of  this  change,  the  organizations  which  were  combining  under  one  authority  structure  were  also 
responsible  for  a  change  from  a  formal  organizational  structure  based  on  functional  groupings,  to  an 
organizational  structure  based  on  IPTs  (Integrated  Product  Teams),  which  are  groups  that  combine  all 
the  functions  necessary  to  complete  a  project  or  support  a  product.  These  changes  were  having  a 
major  impact  on  personnel  in  the  organizations  studied.  Stress,  fear,  and  lowered  morale  were  the 
resultant  outcomes,  ’with  some  organizations  being  more  affected  by  these  attitudes  than  others. 
Members  of  some  organizations  felt  themselves  more  at  risk  than  members  of  other  organizations, 
based  on  the  centrality  of  their  organizational  missions  to  new  (post  cold  war)  Ar  Force  objectives. 
This  resulted  in  some  organizational  cultures  being  more  change  averse  than  other  organizational 
cultures,  due  to  the  members'  association  of  organizational  change  with  personal  risk. 

This  turbulence  had  an  affect  on  the  implementation  of  new  oflSce  information  and  communication 
technologies.  Organizations  which  had  experienced  greater  than  average  turbulence,  especially  if 
members  perceived  a  high  degree  of  association  between  organizational  change  and  personal  risk 
(especially  job  loss),  were  averse  to  technological  change  as  well.  This  can  be  seen  in  the  data,  ■with 
medium  to  medium/high  usage  organizations  experiencing  less  turbulence  and  associating  these 
changes  less  "with  risk,  than  low  to  low/medium  usage  organizations.  High  degrees  of  turbulence  also 
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affected  the  amount  of  time  and  energy  available  to  cope  with  technological  implementation  and 
learning.  Concentration  is  focused  on  the  changes  occurring,  instead  of  implementing,  learning,  and 
using  new  office  technologies.  In  many  instances  it  was  reported  that  the  extent  of  information  and 
communication  technology  usage  was  a  factor  of  how  many  people  w  ere  left  after  downsizing,  and  the 
amount  of  work  required  of  them.  Personnel  with  extremely  high  workloads  (people  doing  the  work 
of  more  than  one  person)  did  not  have  the  time  to  learn  a  new  system,  it  was  easier  and  faster  to 
accomplish  their  work  using  familiar  methodologies.  Learning  how  to  use  a  new  office  technology 
requires  an  investment  in  time  that  is  often  not  available  to  personnel  with  more  to  do  in  a  day  than  can 
be  done.  Organizations  that  had  already  successfully  automated  their  office  work  processes,  and  then 
lost  personnel,  would  not  be  affected  by  this  phenomenon. 


Finding  #8:  Single  Product  SPOs  can  better  implement  new  IT 

Organizational  mission,  as  far  as  acquisition  versus  sustainment,  was  not  correlated  to  usage  levels. 
Acquisition  organizations  had  both  very  high,  and  very  low,  as  well  as  medium,  usage  levels.  This  was 
also  true  of  sustainment  organizations. 

The  formal  organizational  structure  was  almost  identical  at  every  organization  studied.  SPOs  (System 
Program  Offices)  were  two-letter  organizations  and  contained  sub-units  denoted  by  three  digits  (three 
letters  or  two  letters  and  a  number).  These  three-digit  organizational  sub-units  were  broken  down 
further  into  groups  specified  by  four-digits.  Variation  in  usage  levels,  then,  did  not  correspond  to 
variation  in  formal  organizational  structure. 

Variation  in  usage  levels  did  correspond,  however,  to  the  number  of  products  worked  on  by  the 
organization.  Single  program  SPOs  were  more  likely  to  have  medium  to  medium/high  usage  levels 
than  basket  SPOs  or  basket  divisions  of  basket  SPOs.  Basket  SPOs  have  a  number  of  programs  that 
are  usually  autonomous,  requiring  little  or  no  communication  between  product  or  project  groupings. 
Also,  each  program  frequently  has  its  own  special  communication  and  information  technology  needs: 
specific  systems  that  are  necessary  or  unnecessary  for  specific  kinds  of  tasks.  These  qualities  of 
independence,  along  with  the  fact  that  basket  SPO  programs  often  were  responsible  for  their  own 
resources,  and  could  purchase  software  independently  of  the  other  programs  in  the  SPO,  made 
widespread  implementation  of  universal,  communal  information  and  communication  technologies 
difficult.  Furthermore,  usage  levels  will  be  at  a  lower  level  in  these  organizations  simply  because  of  the 
lack  of  need  to  communicate  or  share  information  with  other  SPO  groups. 

As  stated  previously,  teams  in  each  organization  tended  to  be  cross-functional  and  based  on  an  IPT 
structure  due  to  the  change  to  IWSM.  This  did  not  mean  functional  teams  were  non-existent,  but  that 
EPTs  were  forming,  and  becoming  the  majority.  This  was  true  in  most  of  the  organizations  studied, 
and  thus  presence  of  IPTs  could  not  be  found  to  correlate  to  organizational  usage  levels. 
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Finding  #9:  Social  and  physical  infrastructure  affect  IT  implementation 

The  age  of  the  organization  did  correspond  to  usage  levels.  For  the  most  part,  the  pattern  showed  that 
the  older  the  organization  was,  the  lower  the  usage  levels  it  had.  This  can  be  attributed  to 
well-established  practices  of  using  manual  techniques.  Once  manual  techniques  are  perfected,  and  they 
have  been  proven  successful  in  completing  the  required  work,  their  use  becomes  ingrained  in  the 
culture,  and  it  is  much  more  difficult  to  change  to  automated  office  technology.  If  office 
communication  and  information  technologies  are  implemented  before  manual  methods  have  time  to 
become  entrenched,  they  will  be  received  with  less  resistance  and  will  be  used  more  frequently.  A 
young  organization  that  waits  to  implement  office  automation  technology  is  at  risk  of  losing  their 
advantage:  once  manual  methods  are  established  and  proven  successful,  it  is  possible  to  develop  a 
cultural  attitude  of  "if  it  ain't  broke,  don't  fix  it  " 

The  physical  environment  of  the  organization  is  also  correlated  to  usage  levels.  The  less  infrastructure 
already  in  place  in  the  building,  the  less  capabilities  there  are  available,  and  the  longer  implementation 
of  new  technologies  takes. 

Finding  #10:  Perceptions  of  attitudes  affect  receptivity 

Whether  someone  can  be  persuaded  to  accept  and  use  new  office  automation  technology  has  more  to 
do  with  their  perceptions  of  other  people's  attitudes  about  the  technolog>'  than  it  has  to  do  with  their 
indKidual  attitudes  about  the  technology.  If  organizational  members  perceive  others  as  resistant  to 
new  information  and  communication  technologies,  they  are  less  likely  to  use  the  technology.  If  people 
perceive  a  receptive  office  culture,  they  are  more  likely  to  accept  and  use  new  information  and 
communication  technologies,  and  the  opposite  is  true  as  well.  An  individual,  him  or  herself,  may  have 
very  positive  attitudes  about  office  information  and  communication  technologies,  but  if  the 
organizational  culture  portrays  negative  attitudes  and  resistance,  this  individual  is  unlikely  to  buck  the 
system,  so  to  speak.  The  data  collected  supports  the  finding  that  even  champions  can  end  up  as  low 
level  users  with  negative  attitudes  after  being  in  a  low-use  organization  with  high  resistance. 


Finding  #11:  A  moderate  level  of  fragmentation  is  a  facUitator  of  IT  adoption 

A  certain  degree  of  fragmentation  seems  to  be  conducive  to  acceptance  and  usage  of  office  information 
and  communication  technologies.  When  organizational  members  must  cope  with  frequent  TDYs, 
and/or  the  necessity  of  communication  with  remote  locations  (organizational  members  located  out  of 
state),  this  increases  the  likelihood  of  a  high  usage  level  of  communication  technologies  such  as  E-mail 
and  video  teleconferencing.  In  other  words,  collocation  can  actually  hinder  some  implementations, 
instead  of  helping  them.  When  people  can  simply  push  their  chairs  back  and  talk  to  each  other  (or 
communicate  via  the  \TLV  -  Very  Loud  Voice  -  method)  they  will  be  less  likely  to  see  the  need  for 
learning  to  use  office  information  and  communication  technologies.  Communication  within  every 
organization  studied  took  place  on  an  ad-hoc,  as  needed  basis.  There  were  regular  meetings,  but  often 
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a  number  of  people  could  not  attend  these  meetings  because  they  were  on  TDY.  While  most 
communication  took  place  face-to-face  or  over  the  telephone,  communication  technologies  were 
expressed  as  especially  handy  for  leaving  messages  for  people  who  were  out  of  the  office.  The  more 
frequent  the  TDYs  experienced,  the  more  E-mail  was  considered  a  necessity  for  organizational  work. 

On  the  other  hand,  when  different  but  connected  organizations  are  located  far  apart,  the  organizations 
tend  to  drift  away  from  one  another.  Non-collocated  organizations  may  be  at  very  different  levels  of 
usage,  despite  their  connection  and  frequent  communication.  As  each  organization  focuses  on  its  own 
issues  and  needs,  and  develops  its  own  methods  for  accomplishing  the  mission,  the  feeling  of 
connectedness  dwindles,  and  may  even  spark  negative  feelings  toward  other  organizations  connected 
to  them  via  the  command  structure.  This  adds  to  the  variability  between  organizations.  Just  because 
two  organizations  report  to  the  same  central  authority  figure,  it  cannot  be  assumed  that  they  have 
similar  acceptance  and  usage  levels  of  office  automation  technology. 


6.3  Findings  related  to  SPO  sociotechnical  systems 


This  last  set  of  findings,  based  in  sociotechnical  systems  observations,  relates  to  issues  that  a  SPO 
director  has  leverage  over.  Whether  through  training  programs  to  encourage  different  attitudes,  the 
structuring  of  computer  support  groups,  or  the  use  of  incentives  to  overcome  the  legacy  of  previous 
implementation  efforts,  these  findings  point  to  issues  that  are  uniquely  the  concern  of  SPO 
management. 


Finding  #12:  Individual  attitudes  do  not  correlate  with  usage  levels 

Individual  attitudes  towards  office  information  and  communication  technology  are  very  positive  in  low 
usage  organizations,  become  more  negative  in  low  to  medium  usage  organizations,  and  become  more 
positive  again  with  medium  to  medium/high  usage  organizations.  In  each  medium,  and  medium/high 
usage  organization  the  percentage  of  negative  attitude  statements  about  technology  was  almost  as  high 
as  the  percentage  of  positive  attitude  statements.  These  results  actually  correspond  to  the  number  of 
specific  problem  statements  being  reported  by  office  technology  users.  Low  usage  organization 
members  make  few  specific  problem  statements  because  they  use  such  a  small  amount  of  technology. 
Low  to  medium  use  organizations  are  usually  having  implementation  problems,  thus  the  increase  in 
problem  statements,  and  medium  to  medium/high  usage  organizations  are  using  so  many  different 
systems  that  problems  are  not  only  bound  to  appear,  but  are  bound  to  be  noticed  and  commented  on 
because  of  the  organizational  reliance  on  the  technologies  for  the  completion  of  their  work. 

Taken  together,  findings  #7  and  #8  present  an  interesting  conclusion  and  an  interesting  opportunity. 
Our  data  indicate  that  it  is  not  so  much  the  individual's  attitude  toward  the  systems,  as  it  is  his  or  her 
perceptions  of  others'  attitudes,  that  affects  his  or  her  willingness  to  try  new  systems.  This  means  that 
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if  management  can  cultivate  a  culture  of  receptivity,  through  leadership,  example,  communication,  and 
the  isolation  of  anti-champions,  then  the  individual  users  will  accept  the  new  systems. 


Finding  #13:  The  support  group  is  critical 

Perhaps  the  most  important  variable,  with  respect  to  the  success  or  failure  of  office  information  and 
communication  technology  implementation,  is  the  computer  support  provided  before,  during,  and  after 
the  implementation  effort.  This  is  the  key  to  creating  receptivity,  maintaining  receptivity,  and  quashing 
resistance  The  computer  support  group  is  responsible  for  the  history  of  technological  change,  the 
change  process,  the  change  plans,  and  training. 

The  culture  of  the  military  makes  it  such  that  if  the  computer  support  group  is  large,  and  diverse  in 
both  gender  and  ethnic  background,  the  computer  support  group  will  be  more  successful.  Three  of  the 
top  four  organizations  in  office  technology  usage  levels  had  large,  diverse  computer  support  groups. 
The  Air  Force  population  as  a  whole  is  diverse,  and  when  this  is  also  reflected  in  the  computer  support 
group,  personnel  are  more  able  to  find  compatible  personalities  to  aid  them  in  learning  and  using  the 
systems. 

Furthermore,  the  study  indicated  that  the  computer  support  personnel  must  be  knowledgeable  about 
system  usage  as  well  as  system  implementation.  All  four  of  the  top  usage  organizations  had  computer 
support  groups  that  could  assist  users,  not  only  in  getting  their  information  back  if  they  somehow 
deleted  it,  or  getting  back  on-line  if  they  went  off-line,  but  in  day  to  day  questions  about  usage 
methodologies  as  well.  When  computer  support  personnel  do  not  know  what  the  systems  are  used  for, 
or  how  to  use  them,  the  organization  often  reflects  this  in  low-level  usage  patterns. 

Training  is  also  important  in  the  success  of  information  and  communication  technology 
implementations.  Just-in-time  training  was  reportedly  the  most  effective.  Training  too  far  in  advance 
allows  personnel  to  forget  key  information  and  training  after  implementation  allows  personnel  to 
experience  lack  of  usefulness,  both  of  which  are  detrimental  to  receptivity.  Most  of  the  training,  except 
in  the  highest  usage  organizations,  was  not  just-in-time,  but  was  conducted  either  far  in  advance  of 
implementation  or  after  the  fact.  Furthermore,  the  training  must  be  appropriate  to  personnel 
capabilities.  If  training  is  too  basic,  or  too  advanced,  it  will  not  be  usefiil.  Training  that  was  too  basic 
was  indicated  to  be  a  problem  in  some  of  the  organizations  studied. 


Finding  #14:  A  history  of  failed  implementation  is  difTicult  to  overcome 

When  the  history  of  office  automation  technology  implementation  includes  failures,  it  is  much  more 
difficult  to  implement  subsequent  systems  successfully.  Organizational  members  retain  a  memoiy  of 
the  past  failures  and  are  likely  to  expect  this  implementation  effort  to  go  the  way  of  the  other  efforts  - 
down  the  tubes.  Organizations  with  large  numbers  of  resistant  personnel  may  find  that  members  try  to 
simply  wait  it  out,  believing  that  if  they  delay  learning  and  using  the  system  long  enough,  it  will 
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eventually  pass  away  or  be  replaced  by  something  new,  much  as  other  organizational  fads.  We  found 
the  best  predictor  of  future  success  in  implementation  was  a  good  track  record.  Some  organizations 
were  always  stumbling,  while  others  were  consistently  successful  in  their  implementation  efforts. 


Finding  #15:  User  input  facilitates  implementation 

If  the  technological  change  process  does  not  include  user  inputs,  this  reduces  the  likelihood  of  high 
usage  levels.  Three  of  the  top  four  organizations,  in  usage  levels,  actively  solicited  user  input  before 
implementation.  Lower  usage  organizations  did  not  engage  in  this  practice.  The  implementation 
process  is  also  facilitated  by  organizational  member's  knowledge  of  plans  for  technological  change. 
When  personnel  are  aware  of  planned  changes,  they  can  prepare  themselves  for  it,  and  are  not 
surprised  by  the  appearance  of  something  new  on  their  computers.  This  makes  the  change  less  abrupt 
and  eases  the  transition,  reducing  resistance  generated  by  suddenness. 


Finding  #16:  Funding  levels  are  less  critical  than  spending  patterns 

The  amount  of  funding  is  secondary  to  the  enculturated  ways  of  spending  it.  It  was  revealed  during  the 
interviews  that  the  Air  Force  enculturates  its  people  to  spend  money  on  tangible  products,  instead  of 
processes.  Related  to  this,  in  so  far  as  time  equals  money,  time  was  reportedly  not  provided  for 
enticing  new  users,  training,  or  experimentation.  Without  the  time  being  expressly  provided  for  these 
activities,  organizational  members  will  concentrate  solely  on  their  work  tasks,  and  accomplish  them  in 
the  same  way  they  have  always  done  them  -  manually  or  with  minimal  automation. 


Finding  #17:  Security  can  be  a  facilitator  or  an  inhibitor  of  IT  implementation 

Security  requirements  play  an  important  role  in  usage  of  office  information  and  communication 
technologies.  Because  classified  data  must  be  protected,  and  often  is  supposed  to  be  available  only  to  a 
few  organizational  members,  dealing  with  secure  data  produces  special  information  and  communication 
technology  requirements.  When  organizational  members  have  different  levels  of  security  clearances 
and  are  restricted  in  their  communication  with  each  other  about  the  data  they  work  with, 
implementation  of  information  and  communication  technologies  is  made  very  problematic. 
Furthermore,  usage  will  be  curtailed  by  organizational  members'  reluctance  to  spill  the  beans,  so  to 
speak.  Another  problem  identified  in  the  automation  of  secure  offices  is  the  necessary,  but  sometimes 
overzealous,  oversight  by  security  officers.  When  more  than  one  security  officer  (either  internal  to  the 
organization,  or  external  to  it  -  e  g.  from  the  Office  of  Special  Investigations)  has  the  authority  to  make 
decisions  affecting  what  systems  are  implemented  and  how,  implementation  becomes  extremely 
problematic.  Often  there  is  overlap  in  areas  of  responsibility  and  territorial  struggles  between  security 
officers  may  ensue,  delaying  implementation  and  reducing  its  likelihood  of  success.  There  may  also  be 
so  many  security  measures  installed  in  the  system  that  it  makes  it  difficult  to  use,  or  limits  its  usefulness, 
again  inhibiting  acceptance  and  usage. 
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Finding  #18:  Work  processes  have  not  changed  in  connection  with  implementation 

The  successful  implementations  that  we  observed  of  office  information  and  communication 
technologies  were  not  accompanied  by  changes  in  work  process.  It  was  mentioned  that  the  Air  Force 
funds  system  changes,  not  work  process  changes.  Technology  was  not  described,  in  any  interview,  as 
producing  a  change  in  the  way  work  was  accomplished,  nor  did  we  discover  any  descriptions  of  work 
process  analysis  or  change  as  a  precursor  of  systems  acquisition  or  deployment.  When  work  process 
change  was  a  requirement  for  the  successful  implementation  of  new  systems,  the  change  was  perceived 
as  a  barrier;  the  implementation  was  either  not  pursued  or  was  not  successful. 


Finding  #19:  The  contractor  environment  can  either  facilitate  or  inhibit  implementation 


We  found  that  the  contractor  environment  was  in  some  cases  a  significant  issue  for  implementation, 
although  this  tended  to  be  more  a  management  rather  than  a  user  issue.  At  the  user  level  contractor 
relationships  were  described  as  generally  good;  comments  on  contractors  as  driving  implementation  or 
resisting  it  came  from  division  chiefs  and  SPO  directors.  One  program  manager  described  a 
contractor/customer  environment  that  was  a  clear  inhibitor  of  systems  implementation:  a  dual-sourced 
avionics  device,  with  a  third  contractor  as  systems  integrator,  more  than  a  dozen  aircraft  ("customers") 
that  it  was  required  to  fly  on,  a  half-dozen  other  devices  that  it  was  required  to  interface  with,  and 
numerous  integration,  calibration,  and  data  communication  problems.  In  this  complex  environment., 
critical  issues  tended  to  revert  to  the  lowest  common  denominator  of  face-to-face  meetings  for  then 
resolution. 

More  generally,  USAF  and  industry  experience  has  shown  that  enterprise  integration  applications, 
whether  CALS,  EDI,  or  the  concurrent  processing  of  engineering  changes  using  a  shared  CAD 
database,  work  best  as  point-to-point,  bilateral  solutions.  A  multilateral  environment,  such  as  that  of 
the  avionics  program,  was  highly  resistant  to  integration.  This  was  brought  out  by  our  data:  the 
single-program  SPOs  had  the  highest  level  of  systems  implementation. 


6.4  Negative  or  counterintuitive  findings 


There  were  three  negative  (or  counter-intuitive)  findings  that  also  bear  mention.  These  are  patterns 
that  we  expected  to  find,  but  were  not  brought  out  by  the  data.  The  first  of  them  is  that  the  ratio  of 
anti-champions  to  champions  was  not  found  to  be  correlated  with  usage  levels  in  the  organizations 
studied.  More  important  is  that  the  computer  support  personnel  are  active  champions,  and  that  none  of 
the  anti-champions  had  enough  authority  to  prevent  systems  acquisition  altogether. 
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Additionally,  organi2ational  prestige  could  not  be  correlated  with  implementation  and  usage  levels  of 
office  information  and  communication  technologies:  some  extremely  low  prestige  organizations  had 
very  high  usage  levels  while  some  extremely  high  prestige  organizations  had  veiy  low  usage  levels. 
This  variable  was  convoluted  because  reasons  for  levels  of  prestige  were  based  on  many  different 
criteria  including:  age,  stage  of  product  development,  mission,  budget,  and  security  levels.  Those 
criteria  were  inconsistent  with  each  other  for  each  organization,  and  usage  levels  were  more  dependent 
on  other  variables  than  on  these.  Although  organizational  prestige  is  an  important  dimension  of  a 
sociotechnical  system,  its  effect  on  the  adoption  of  new  information  technology  is  far  from  linear, 
reflecting  the  multi-dimensionality  of  the  phenomenon. 

Finally,  relationships  with  the  contractors  were  overwhelmingly  described  as  good.  It  must  be 
remembered  that  the  reported  contractor  relationships  took  place  on  an  individual,  personal  level, 
mostly  among  non-management  personnel.  It  was  indicated  that  it  is  the  high-level  management 
relationships  that  are  strained,  if  any  at  all,  not  the  relationships  between  lower-level,  non-management 
workers.  Because  external  relations  were  considered  good  in  almost  every  organization  studied,  there 
can  be  no  correlation  made  between  contractor  relations  on  the  one  hand  and  implementation  and 
usage  of  office  information  and  communication  technologies  on  the  other. 


6.5  Review  of  guiding  hypotheses 


These  findings  in  large  part  support  our  original  guiding  hypotheses.  We  emphasize  again  that  our 
purpose  was  not  to  test  these  hypotheses,  but  rather  to  use  them  to  guide  an  empirical  investigation. 
Had  our  results  been  completely  orthogonal  with  respect  to  the  thirteen  guiding  hypotheses,  then  the 
study  would  have  been  misguided.  Instead,  as  summarized  in  the  table  below,  our  findings  were 
consistent  with  and  further  developed  these  guiding  h5potheses: 


Guiding  hypothesis  ... 


Resuit 


#1 :  Effect  of  program  stage 

#2:  Volume  of  paperwork 

#3:  Stable  funding  promotes  adoption 

#4;  Turbulence  impedes  adoption 

#5:  Poor  supplier  relations  impede  adoption 

#6;  Mission  critical  activities  less  interested  in  OA 

#7:  Effect  of  installed  advanced  IT 

#8:  Positive  user  attitudes  promote  usage 

#9:  Basket  SPOs  less  likely  to  adopt 

#10:  More  recent  organizations  more  likely  to  adopt 

#1 1 :  Organizational  prestige  affects  adoption 

#12.  Effect  of  discipline 

#13:  Implementation  plan 


supported 

insufficient  data 

data  inconclusive 

all  SPOs  highly  turbulent 

not  an  issue  at  user  level 

insufficient  data 

supported 

refuted 

supported 

supported 

supported,  but  not  linear  relationship 
appears  to  have  no  relationship 
supported 
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6.6  Success  stories  and  recommendations  of  the  MIS  managers 

From  the  success  stories  collected  from  CALS  vendors  and  various  AFMC  components,  certain 
patterns  emerge  as  underlying  successful  implementation.  Among  these  are: 

•  Comprehensive  planning 

•  Anticipation  of  anti-champions  and  cultivation  of  champions 

•  Examination  of  work  processes  and  routines 

•  Adequate  training  and  support 

•  Aggressive  implementation  strategies  work  best 

•  Contractor  input  is  valuable 

•  User  input  is  essential 

After  the  fieldwork  for  the  FRAMEAVORK  tool  had  been  completed,  the  team  was  given  the 
opportunity  to  test  the  findings  and  the  model  in  an  actual  implementation  situation.  The 
Reconnaisance  SPO  (ASC/RE)  was  planning  the  implementation  of  several  new  systems,  including 
video  teleconferencing,  EDI,  and  the  sharing  of  CAD  files  with  contractors.  With  the  concurrence  of 
the  SPO  Director,  the  team  used  the  interview  protocols  built  into  the  tool  to  collect  cultural  and  user 
attitude  data  at  the  SPO.  The  entire  procedure  required  approximately  four  hours  of  interviewing  with 
five  SPO  personnel.  The  exact  protocol  was  used  in  an  identical  manner  to  an  on-line,  computerized 
assessment. 

Subsequent  to  the  assessment,  the  team  provided  to  the  same  SPO  personnel  an  alpha  version  of  the 
recommendations.  These  recommendations  were  subsequently  packaged  into  the  software  tool.  This 
report-out  included  both  an  assessment  of  the  leading  implementation  issues,  and  recommendations  for 
managing  the  issues.  Discussion  with  and  comments  from  the  IS  manager  at  ASC/RE,  indicated  that 
the  recommendations  were  considered  to  have  definite  value  in  planning  their  implementation  of  these 
systems. 

Additionally,  when  we  queried  the  SPO  MIS  managers  in  AFMC,  we  found  some  common  themes 
that  from  their  experience  led  to  successful  implementation.  Six  patterns  came  out  of  the  discussion. 
The  patterns,  illustrated  with  direct  quotations  from  MIS  directors  in  the  SPOs,  are: 

Communication  with  and  among  users  is  important.  The  users  do  not  completely  know 
the  requirement  for  the  new  system.  The  communication  among  the  users  is  important.  We 
have  to  hold  the  requirement  meeting  regularly  in  order  to  consolidate  all  the  requirements. 

Old  systems  must  be  abandoned.  Get  users  to  give  up  the  old  method  of  doing  business. 
You  must  give  a  deadline  of  when  the  old  software  will  no  longer  be  available  and  stick  to  it. 
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We  had  users  using  Peachtext  a  year  after  MS  Office  was  introduced  because  we  failed  to  take 
it  away  from  them.  With  the  transition  from  AMS  to  a  local  Email  system  we  are  giving  clear 
deadlines  and  sticking  to  them. 

Leadership  must  create  a  positive  attitude.  There  is  a  natural  resistance  to  change  in  one's 
work  processes,  so  you  need  to  convince  a  user  that  the  new  process  has  value  to  them,  such 
as  taking  less  time  to  accomplish  their  tasking,  etc.  It  goes  back  to  communication  prior  to 
implementation.  You  have  to  create  the  positive  attitude  in  the  user  in  order  to  aid  in  the 
success  of  the  system  implementation.  There  is  a  reluctance  of  many  to  share  their  data  with 
others  in  the  organization  because  it  opens  them  up  to  review  by  others,  such  as  in  a  program 
integrated  schedule.  This  is  difficult  to  overcome  and  sometimes  just  takes  time  to  allow  the 
process  to  evolve.  Up  front  communication  persuades  the  users  of  the  benefits  of  the  system, 
which  overrides  their  reluctance  to  open  their  project  work  to  the  review  of  others. 

A  critical  factor  in  successful  implementation  of  any  information  technology  is  positive  support 
from  the  top  of  the  organization,  whether  or  not  he/she  is  considered  the  champion. 

You  MUST  have  buy-in  from  the  Front  Office  and  Division  Chiefe  first.  You  have  to  show  the 
benefits  to  be  derived  by  having  a  standard  software  suite  and  be  able  to  identify  the  pitfalls  if 
not  followed.  A  precedent  has  been  set  that  can  lead  credence  to  others. 

The  implementation  project  must  have  a  realistic  schedule  and  realistic  capabilities.  As 

with  any  new  initiative,  a  plan  has  to  be  developed  to  ensure  the  success  of  it.  Below  is  a  list  of 
those  factors  deemed  critical  (necessary)  for  success.  1 .  The  concept  of  the  project  must  be 
SOLD  to  managers  and  end-users.  2.  Don't  make  claims  to  capabilities  beyond  what  is 
actually  possible.  3.  Develop  a  reasonable/realistic,  but  conservative,  schedule.  4.  Provide 
regular  updates  on  the  status  of  the  project.  5.  If  it  is  required,  provide  TRAINING! ! ! !! !  6. 
Don't  work  autonomously! ! 

Creating  leadership.  One  of  the  critical  success  factors  I  have  learned  is  to  advertise  the  new 
system.  Let  the  knowledgeable  people  introduce  the  system  to  the  user  and  list  all  the  benefits 
to  convince  the  user.  If  it  is  possible,  arrange  demos  to  the  users  as  much  as  possible.  When 
the  user  starts  building  the  confidence  on  that  system,  they  will  feel  more  comfortable  in  using 
them.  Of  course  the  training  plays  a  important  role  in  it. 

Support  and  maintenance  are  essential.  It  is  essential  to  the  success  of  the  information 
system  that  the  proper  level  of  support  for  maintenance  and  user  training  be  available  on-site  to 
respond  to  needs  that  are  always  time  sensitive.  The  system  must  be  on-line  or  the 
organization's  productivity  suffers. 

In  sum,  these  lessons  and  experiences  offer  clear  guidance  for  any  MIS  manager  planning  an 
implementation  of  new  information  systems. 
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7.  USAF  Applications 


In  the  course  of  the  FRAMEAVORK  project,  the  application  of  FRAMEAVORK  and  its  embedded 
concepts  was  pursued  for  two  distinct  areas:  (1)  portable  maintenance  aids,  and  (2)  DoD  directives  for 
systems  development  and  implementation.  The  results  of  these  two  inquiries  are  presented  here; 


7. 1  Integrated  Electronic  Technical  Manuals  (lETMs)  and  Portable  Maintenance 
Aids  (PMAs) 


In  recent  years  the  Air  Force  has  made  significant  investments  in  the  development  of  Portable 
Maintenance  Aids  (PMAs).  These  are  rugged,  portable,  battery-powered  computers  in  which  can  be 
stored  the  Tech  Orders  and  other  documentation  required  for  aircraft  maintenance.  In  the  traditional, 
paper-based  environment,  the  aircraft  maintainer,  usually  working  out-of-doors  on  the  flight  line,  had 
one  or  several  manuals  opened  beside  his  job.  If  a  new  problem  was  discovered,  he  had  to  make  a  trip 
back  to  the  hangar  for  the  necessary  manual.  If  he  was  working  on  several  interfacing  systems,  perhaps 
as  many  a  dozen  manuals  would  be  arrayed  around  him. 

Conceptually,  with  a  PMA  all  of  this  paper  can  be  replaced  with  electronic  storage  and  a  LCD  display 
Some  of  the  earliest  PMAs,  however,  were  little  more  than  "page  turners",  containing  rasterized 
images  page  by  page  of  the  technical  manuals.  This  format  was  more  awkward  and  diflBcult  to  use 
than  the  paper-based  manual,  and  was  not  speedily  accepted  on  the  flight  lines.  More  recent  PMAs 
have  considerably  more  intelligence  built  into  them:  using  hot  buttons,  hypertext,  and  a  relational 
database,  together  with  augmented  storage  capacity  (up  to  1Gb),  these  more  advanced  tools  potentially 
represents  an  improvement  over  paper.  Future  integration,  through  RF  links,  with  the  CAMS  database 
should  further  improve  their  attractiveness. 

In  preparation  for  a  presentation  on  the  use  of  FRAMEAVORK  as  part  of  a  lETM  implementation,  we 
studied  the  cultural  issues  of  the  typical  flight  line.  Particularly  for  enlisted  grades,  there  are  a  number 
of  cultural  and  organizational  issues  that  given  the  right  combination  of  circumstances  could  interfere 
with  the  effective  use  of  PMAs. 

The  central  issue  is  the  means  of  supervision  and  control  on  the  flight  line.  The  maintainer  is  supposed 
to  have  his  manuals  right  in  front  of  him,  open  to  the  pages  he  is  working  on,  at  all  times.  Quality 
Control  personnel  will  check,  at  least  once  per  shift,  to  assure  this.  If  the  book  is  not  open  to  the  right 
page,  the  maintainer  will  be  flagged  by  QC  Open  book  reference  may  not  be  supported  by  an  lETM, 
and  this  sort  of  visual  supervision  may  be  more  difficult  with  an  LCD  screen. 
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Related  to  this  is  the  status  order  of  the  typical  flight  line.  At  the  top  is  the  aircraft  itself,  as  a  totemic 
object;  just  below  the  aircraft  is  its  pilot,  and  below  the  pilot  is  the  alert  crew.  Shop  personnel  are 
below  the  alert  crew,  and  supply  and  depot  personnel  are  below  them.  The  status  differences  between 
officers  (pilots  and  managerial  personnel)  on  the  one  hand  and  enlisted  (supervisors,  maintainers, 
technicians  and  clerks)  on  the  other  may  lead  to  distrust.  Maintainers  and  bench  technicians  have 
developed  informal  ways  to  adapt  to  this  status  difference. 

To  the  extent  any  integrated  maintenance  support  system  infiinges  on  these  zones  of  autonomy  (of  the 
crew  chief,  supervisor,  or  bench  technician,  and  potentially  others  in  the  depot),  it  may  be  resisted  or 
manipulated  and  evaded. 

Given  this,  we  found  based  on  our  study  of  this  culture  that  four  of  the  management  issues  that 
FRAMEAVORK  examines  have  a  direct  bearing  for  the  flight  line.  These  are: 


History  of  system  implementation  efforts:  previous  experience. 

Turbulence  of  the  work  environment,  particularly  for  higher  management  levels 
Structure  of  the  organization  and  barriers  of  distrust 
Job  design  for  enlisted  personnel  on  the  flight  line. 

In  its  current  form,  the  FRAMEAVORK  tool  provides  an  assessment  of  these  (as  well  as  other) 
management  issues;  a  streamlined  version  would  provide  the  same  assessment  in  a  far  less  intrusive 
manner. 


7.2  Integration  with  Systems  Development  Procedures 


Task  7  of  the  FRAMEAVORK  project  consisted  pf  the  review  of  applicable  Military  Standards  and 
procedures  for  systems  implementation,  to  identify  those  elements  to  which  the  FRAMEAVORK 
architecture  and  concepts  have  relevance.  In  fulfillment  of  this  task,  the  team  reviewed  the  follovvong 
documents: 
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MilStd2167 
MilStd2168 
MilHdbk  8020.1 

MilHdbk  8320.1-M,  8320.1-M-l,  and  8320.1-M-x 


Analysis  of  2167  and  2168 

These  documents  set  standards  for  the  quality  of  software  and  its  documentation.  The  only  standard 
given  for  implementation  of  software  or  systems  is  on  page  2  of  2168,  section  4.5,  where  the 
contractor  is  charged  with  implementing  the  software  in  accordance  with  the  SCPP  for  the  duration  of 
the  contract.  This  would  permit  the  specification  of  the  use  of  FRAMEAVORK  during  the 
implementation  phase  of  a  software  development  program.  There  are  no  human  issues  or  cultural 
standards  identified  or  addressed  in  2167  or  2168. 


Analysis  of  8020.1 

8020.1  gives  guidance  in  process  improvement  projects  required  for  any  information  technology 
implementation.  This  appears  to  be  the  best  place  to  try  to  include  FRAMEAVORK  as  a  standard. 

The  policy  in  this  manual  states  (pages  2-3)  that  "important  gains  in  functional  and  information 
management  effectiveness  and  efficiency  can  be  achieved  by  continued  evaluation  and  restructuring  of 
the  way  DoD  missions  are  accomplished  and  supported  .  .  .  using  integrated  and  standard  processes. 
Sound  business  practices  will  be  used  to  achieve  these  objectives." 

A  readiness  assessment  such  as  that  provided  by  FRAMEAVORK  is  part  of  sound  business  practice 
and  can  be  an  integral  part  of  evaluation  and  restructuring. 

Page  3:  "[Specific  personnel]  will  evaluate  processes,.  .  .  and  implement  simplified,  streamlined, 
standardized,  and  cost  effective  alternatives  to  current  processes  across  the  DoD.  Integrated  cross¬ 
functional  approaches  shall  be  sought  wherever  possible." 

FRAMEAVORK  is  a  simplified,  streamlined,  and  cost-effective  alternative  to  implementing  new 
programs  and  systems  without  taking  into  account  human  issues  and  cultural  factors  that  may  have  a 
negative  impact  on  use  of  the  new  systems. 

Page  5:  "To  reduce  risk,  functional  process  improvements  shall  be  conducted  through  the  rapid 
implementation  of  incremental  and  evolutionary  improvements." 


Wizdom  Systems,  Inc. 


FRAMP'vAVORK  Final  Report,  page  65 


December  11,  1995 


FRAMEAVORK  will  help  reduce  risk  and  will  allov/  rapid  implementation  of  incremental  and 
evolutionary  improvements  to  be  carried  out  with  a  minimum  of  lost  time. 

Page  7:  "[Specific  personnel]  will  establish  and  execute  internal  procedures  to  routinely  identify, 
evaluate,  justify,  and  implement  fiinctional  process  improvements  by  ensuring  the  application  of  sound 
business  practices  (and  IM  principles  and  policies) . . . 

Page  7:  Acquire  on  a  fee-for-service  basis  and  manage  interdisciplinary  functional  and  technical 
resources  to  prepare  plans,  analyses,  studies,  and  evaluations  of  current  and  proposed  processes,  data, 
and  [Automated  Information  Systems] 

Page  11:  "[Specific  personnel]  shall  develop  standard  methods  and  tools  to  support  the  conduct  of 
functional  process  improvement,  provide  training  in  their  use,  and  provide  technical  assistance  in  all 
aspects  of  Defense  IM  program  implementation. 

These  paragraphs  offer  justification  for  using  FRAMEAVORK  and  assigning  specific  personnel  the 
responsibility  for  its  administration  and  implementation. 

Page  12:  "...  Strategic  plans  shall  be  used  to  document  and  manage  execution  of  process  improvement 
within  each  functional  area 

FRAMEAVORK  can  be  incorporated  as  part  of  any  strategic  plan  that  documents  and  manages  the 
execution  of  process  improvement. 

Pages  3-1  and  3-2:  FRAMEAVORK  fits  easily  into  the  concept  delineated  here  that  defines  functional 
process  improvement  and  discusses  the  performance  of  process  improvement.  Especially  noteworthy 
is  the  discussion  of  interdisciplinary  teams  that  conduct  process  improvement  analysis,  evaluation, 
justification,  planning,  and  implementation.  Each  of  these  activities  would  be  enhanced  by  using  the 
data  and  strategies  supplied  by  FRAMEAVORK. 


Analysis  of  8320. 1-M,  8320. 1-M-l,  and  8320. 1-M-x 

These  manuals  deal  with  (1)  data  administration  procedures,  (2)  data  element  standardization,  and  (3) 
DoD  data  model  development,  approval,  and  maintenance  procedures. 

DoD  8320  1-M  is  the  most  relevant  to  the  use  of  FRAMEAVORK. 

The  area  where  FRAMEAVORK  would  be  most  useful  is  in  the  development  of  the  annual  DoD  Data 
Administration  Strategic  Plan  (DASP)  that  will  be  used  to  define,  plan,  implement,  and  operate  the 
DoD  Data  Administration  Program  (DAP).  The  mission  of  the  DAP  is  "to  provide  for  effective, 
economic  acquisition  and  use  of  accurate,  timely,  and  shareable  data  to  enhance  mission  performance 


Wizdom  Systems,  Inc. 


FRAMEAVORK  Final  Report,  page  66 


December  1 1,  1995 


and  system  interoperability."  Annual  Planning  Guidance  is  developed  by  the  DoD  Data  Administrator 
and  distributed  to  the  components  and  OSD  PSAs  to  assist  in  the  preparation  of  their  strategic  plans. 

FRAMEAVORK  could  be  used  to  determine  human  issues  and  cultural  influences  that  should  be  taken 
into  account  when  preparing  the  strategic  plans. 

Provision  is  made  to  provide  education,  training,  and  consultation  services  to  improve  understanding, 

communication,  and  acceptance  of  new  roles  and  responsibilities.  Provision 

is  also  made  to  acquire  resources  required  to  implement  the  data  administration  action  plans. 

These  paragraphs  justify  the  acquisition  of  FRAMEAVORK. 
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8.  Commercialization  plans 


At  Wizdom  Systems,  Inc.,  the  FRAMEAVORK  initiative  represented  an  important  continuation  of  the 
company's  efforts  in  understanding  the  human  side  of  technology.  The  successes  achieved  with  the 
tool  development  have  spurred  an  interest  in  its  further  commercial  development. 

Failures  of  organizations  to  successfully  reengineer  their  business  processes  and  implement  new 
technologies,  purported  to  be  as  high  as  75%,  have  been  widely  publicized  in  recent  months.  These 
failures  are  typic^y  attributed  to  soft  factors,  such  as  lack  of  management  commitment,  insufficient 
planning,  and  organizational  resistance  to  change.  This  latter  category  is  currently  given  an  enormous 
amount  of  attention  in  industry  conferences,  publications,  and  professional  societies  as  a  major  factor 
contributing  to  reengineering  failures.  It  is  widely  agreed  among  these  sectors  that  what  is  needed  is  a 
way  to  anticipate,  prepare  for,  and  overcome  the  social,  organizational,  and  cultural  barriers  that 
impact  successful  implementation  of  change.  The  FRAMEAVORK  tool,  extended  to  include  cultural 
models  and  pattern  recognition  sets  for  specific  government,  and  commercial  organizations/industries 
can  provide  a  means  to  do  exactly  this.  Wizdom  intends  to  extend  and  commercially  deploy  the 
FRAVIEAVORK  tool  initially  in  the  automotive  and  healthcare  industries  and  secondly  in  other 
government  and  manufacturing  sectors. 

For  each  organization/industry  specific  model  it  will  be  necessary  to  validate  the  variables 
and  re-calibrate  the  model  to  account  for  the  new  variable  set.  Several  of  the  variables  in  the 
FRAMEAVORK  tool  are  not  unique  to  SPOs  and  ALCs.  Numerous  corporations  are  characterized  by 
fragmentation  and  turbulence,  and  resource  scarcity  is  an  abiding  characteristic  of  most  government 
and  commercial  organizations.  Although  some  variables  are  unique  to  the  Air  Force,  others  can  be 
generalized  to  multiple  environments,  and  the  model  adjusted  accordingly.  Therefore,  developing  the 
FRAMEAVORK  model  for  government  environments  requires  a  validation  of  the  variables  in  the 
specific  environment,  an  identification  of  any  potentially  new  variables,  and  recalibration  of  the  model 
to  account  for  the  new  variable  set. 

Wizdom  is  initially  pursuing  an  adaptation  of  the  FRAMEAVORK  tool  to  the  automotive  and 
healthcare  industries  in  conjunction  with  two  efibrts  Wizdom  is  currently  supporting.  Following 
modification  of  the  tool  to  these  two  specific  industry  segments,  Wizdom  will  aggressively  market  and 
sell  the  tool  to  this  community  of  users.  The  following  paragraphs  describe  the  efibrts  which  will 
provide  the  initial  industry  specific  domains  for  FRAMEAVORK  expansion. 
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8. 1  Automotive  Applications 


The  Manufacturing  Assembly  Pilot  project,  now  in  its  second  phase,  has  achieved  an  adequate  level  of 
understanding  the  relationships,  processes  and  communication  characteristics  of  the  automotive  seating 
supply  chain.  As  a  result,  the  study  has  generated  a  set  of  recommendations  to  be  implemented  during 
1995  at  sixteen  automotive  firms.  Each  project  participant  recognizes  that  change  management  is 
going  to  be  as  important,  or  more  important  than  any  technology  deployment,  and  each  firm  has  agreed 
to  commit  as  much  resources  as  required  to  see  this  initiative  succeed.  Wizdom  plans  to  modify/extend 
the  FRAMEAVORK  variable  set  for  this  industry  and  prototype  the  new  product  in  each  of  these 
sixteen  automotive  firms.  Feedback  fi-om  prototype  deployments  will  be  used  to  modify/tune  the 
industry  variable  set,  following  which  the  product  will  be  marketed  to  the  automotive  industry  at  large. 


8,2  Healthcare  applications 


The  Health  Informatics  Initiative  (HII)  is  a  visionary,  systematic  approach  to  using  health  information 
applications  to  tackle  many  of  the  nation's  healthcare  problems.  Information  technology  plays  a  key 
role  in  healthcare  by  streamlining  paperwork  and  improving  access  to  health  information  by 
communities,  patients,  hospitals,  clinics,  schools,  public  health  organizations  and  research  institutions. 
Hn  is  a  project  jointly  funded  by  industry  and  the  Department  of  Commerce  Advanced  Technology 
Program  (ATP)  and  administrated  by  the  National  Institute  of  Standards  and  Technology  (NIST). 

A  major  objective  of  MI  is  to  help  define  and  promote  health  standards  to  facilitate  a  seamless 
integration  of  applications,  tools,  and  databases.  The  new  integrated  toolset  based  on  a  common 
database  of  objects  will  allow  complex  systems  modeling  to  be  achieved  faster  and  more  accurately. 
The  application  of  state-of-the-art  health  information  reengineering  tools  developed  in  this  initiative  and 
implementation  of  the  changes  they  bring  about  will  necessitate  the  same  need  to  assess  and  manage 
social,  organizational  and  cultural  change  being  experienced  in  other  industries.  The  FRAMEAVORK 
tool  can  fill  this  need. 

The  first  phase  of  the  MI  project  will  provide  three  specific  new  products  for  the  Healthcare  Industry. 
The  first  product  is  an  extended  version  of  the  FRAMEAVORK  tool  to  evaluate  the  human  factor 
issues  in  this  industry.  FRAMEAVORK  will  be  adapted  for  healthcare  specific  applications  and 
prototyped  as  part  of  this  project.  As  with  the  automotive  version,  prototype  feedback  will  serve  to 
modify  and  fine  tune  the  tool,  after  which  it  will  be  aggressively  marketed  to  the  healthcare  industry. 

Before  commercialization  of  the  FRAMEAVORK  tool  for  healthcare  and  automotive,  further 
validation  and  field  trials  of  the  existing  tool  are  desirable.  In  addition  to  its  commercial  work,  Wizdom 
currently  supports  several  DoD  CIM  initiatives  which  could  benefit  fi'om  the  human  &ctors  and 
cultural  assessment  features  offered  by  FRAMEAVORK.  Wizdom  will  offer  on-site  assistance  for 
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nominal  remuneration  to  any  DoD  component  or  other  government  agency  that  wants  to  use  the  tool 
with  a  target  of  20  different  user  organizations.  The  feedback  from  these  field  trials  will  be  factored 
into  design  modifications.  Following  rollout  of  the  FRAMEA¥ORK  tool  for  automotive  and 
healthcare,  Wizdom  will  select  the  government  or  commercial  domam(s)  for  the  next  product 
adaptation. 

In  addition  to  supplementary  field  trials,  commercialization  of  the  FRAMEAVORK  tool  requires  three 
general  design  tasks  followed  by  seven  domain  specific  tasks.  The  three  general  design  tasks  are: 

•  Restructure  database 

•  Reformat  knowledge  base 

•  Design  for  easy  plug-in  knowledge  bases 

For  each  industry  specific  rollout  Wizdom  will  perform  the  following  tasks, 

•  Determine  domain  variables 

•  Modify  variables 

•  Engineer  Knowledge  Base 

•  Recalibrate  tool  to  new  variable  set 

•  Prototype  tool  in  new  domain 

•  Incorporate  prototype  feedback 

•  Market  domain  product 

Marketing  wiU  be  accomplished  through  the  same  channels  used  by  Wizdom  to  market  its  current 
products.  Namely,  advertising  in  industry  related  publications,  attending  industry  trade  shows,  direct 
mailings  to  select  databases,  and  featuring  the  new  products  in  Wizdom  sales  collateral.  Additionally,  a 
module  will  be  added  to  the  Wizdom  training  curriculum  providing  training  in  the  new  tool.  This 
training  will  be  featured  at  Wizdom's  monthly  executive  seminars,  which  will  be  tailored  and  promoted 
within  the  industry  where  the  tool  is  currently  being  marketed. 
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9.  Conclusion: 

Recommendations  and  Areas  for  Future  Research 


The  findings  of  this  research  have  broad  implications  for  the  deployment  of  new  information 
technology  within  AFMC.  These  are  evident  in  the  findings  presented  in  section  6.  Rather  than  review 
these,  our  conclusion  and  recommendations  will  focus  on  the  types  of  research  and  tool  development 
that  our  research  found  necessary. 


9.1  Development  concept 


In  the  course  of  our  research  we  discovered  that  management  culture  was  a  unique  and  independent 
variable.  That  is  to  say,  in  the  process  of  systems  implementation,  the  attitudes  of  the  manager  have  a 
significant  effect  independent  of  both  the  technology  and  the  culture  of  the  organization.  This  insight 
led  us  to  create  a  tool  that  would  advise  and  orient  management,  rather  than  provide  directive  findings; 
and  further,  it  underlaid  the  design  of  the  report-out  screen  (figure  8);  the  thermometer  on  the  right- 
hand  side  of  the  screen  shows  our  conclusion  having  takentfie  temperature  of  the  organization  in  terms 
of  overall  implementation  health.  Every  manager  we  briefed  on  our  results  inside  his  SPO  was 
interested  in  this. 

Based  on  the  receptivity  we  found  to  issues  like  this,  and  again  with  perfect  hindsight,  we  conclude 
that: 


The  next  generation  of  readiness  assessment  tools  should  he  driven  by  an  understanding  of 
management  adtiire. 

The  great  strength  of  the  FRAME, WORK  tool  is  in  the  understanding  and  insight  it  provides  into  user 
cultural  issues.  Yet  unless  these  are  presented  and  packaged  in  terms  of  the  managers'  work  process 
and  concerns,  the  manager  will  find  them  more  academically  interesting  than  practically  useful.  This 
leads  us  to  our  first  recommendation: 

1.  A  study  of  management  culture,  attitudes  and  work  processes  should  be  a  leading 
task  in  the  development  of  cultural  tools. 

The  FRAMEAVORK  tool  would  gain  further  strength  through  pilot  testing  and  calibration  with  the 
personnel  and  functions  that  have  a  decisive  impact  on  technology  implementation,  including: 

Laboratories  developing  technology 
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Staff  functions  supporting  the  technology 

Vendors  providing  the  technology 

Prime  contractors  promoting  or  resisting  the  technology 

AFMC  and  Air  Staff  champions  and  initiatives 

AFMC  and  Air  Staff  supporting  or  conflicting  priorities 

We  previously  stated  that  an  early  finding  of  our  study  was  that  the  bottlenecks  to  CALS 
implementation  seemed  to  be  not  so  much  in  the  SPOs  and  ALCs  as  at  these  higher  levels.  Yet  our 
only  visibility  into  these  was  through  the  eyes  of  the  SPOs  and  to  a  lesser  extent  the  ALCs.  Our 
second  recommendation  is  therefore: 


2.  Further  study  of  technology  implementation  should  be  made  within  the  context  of  an 
overall  Air  Force  technology  implementation  model. 

Although  such  models  exist  describing  a  rational,  idealized  {AS-OUGHTA-BE)  process,  we  lack 
models  that  describe  the  real,  AS-IS  process  with  all  of  its  ambiguities,  conflicting  interests,  hidden 
agendas,  and  performance  shortfalls.  Yet  without  a  model  of  the  real  (rather  than  idealized)  context, 
further  studies  of  organizational  readiness  will  simply  yield  idealized  (rather  than  real)  findings. 
FRAMEAVORK  finessed  this  by  concentrating  solely  on  the  system  user. 


9.2  Development  methods 


In  our  original  project  plan,  beta  deployment  occupied  the  last  three  months  of  a  17-month  project 
schedule.  This  is  approximately  proportional  to  the  percentage  of  project  resources  devoted  to  the 
beta  test,  as  illustrated  in  the  pie  chart  on  page  39.  Although  engineering  development  delays  and 
support  delays  stretched  this  out,  the  general  picture  of  engineering  development,  followed  by  field 
test,  foUowed  by  refinements  and  final  release,  was  largely  adhered  to.  This  foreclosed  what  could 
have  been  an  opportunity  to  incorporate  management  culture  into  tool  development.  Hence  our  final 
recommendation: 

3.  Science-driven  projects  should  devote  at  least  one-third  of  project  resources  to 
phased  beta  deployment,  with  one-third  to  one-half  of  the  development  effort  between 
two  or  three  beta  tests. 

The  model  proposed  here  is  a  management  concept  to  assure  that  beta  test  is  an  integral  part  of  the 
development  discipline.  This  will  still  permit  systems  to  incorporate  new  scientific  discoveries,  as  we 
have  done  here;  yet  it  will  increase  the  likelihood  that  these  discoveries  will  be  used,  once  the  system  is 
released. 
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Appendix  B:  Interview  Protocols 


There  are  five  interview  protocols: 

I :  Director  or  Deputy  Director 

II;  Division  Chief  or  Branch  Manager  (or  Deputy) 

III:  Non-Management  Personnel 

IV:  Rapid  Assessment  Focus  Groups 

V:  Computer  Support 


Interview  Protocol  I:  Director  or  Deputy  Director 

1.  Self  biography; 

Name 

Function 

Management  style 

Computer  training  (inside  and  outside  AF) 

2.  Describe  the  organizational  structure  of  the  SPO,  and  the  type  of  work  handled. 

How  many  personnel  are  in  the  SPO? 

Physical  location  of  SPO  personnel? 

What  are  the  sizes  of  the  work  groups  in  which  interviewing  will  take  place? 

Is  it  a  single  program  SPO  or  a  basket  SPO? 

Are  teams  or  used  extensively?  How  are  they  used?  Are  teams  matrixed? 

Are  organizational  divisions  or  branches  composed  of  one  or  multiple  disciplines? 

If  both  functional  units  and  matrixed  teams  are  used,  which  manager  has  more  authority  over 
personnel? 

3.  What  is  the  approximate  level  of  funding  for  the  SPO  on  an  annual  basis?  Has  this  changed  over  the 

past  three  years?  If  so,  how? 

4.  Does  this  SPO  deal  with  any  classified  (or  SAR)  data? 

If  so,  what  percentage  of  the  work  is  secure? 

What  is  the  relationship  between  security  and  oflfice  computer  technology  use? 

5.  At  what  program  stage  of  the  product  development  lifecycle  is  the  SPO  currently  working  in  most 

intensively:  concept,  engineering,  development,  production,  deployment,  recycling? 

6.  How  does  this  SPO  compare  with  others  in  obtaining  the  type  of  personnel  that  it  needs? 
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7.  Describe  the  changes  that  have  taken  place  in  the  SPQ  over  the  past  two  years. 

Organizational  structure;  Leadership;  Personnel;  Location;  Technology 

8.  What  has  been  the  SPO's  history  of  office  computer  technology  use? 

What  changes  are  projected  for  new  computer  technology  in  the  future? 

What  is  the  process  by  which  new  technology  is  introduced  into  the  SPO? 

What  are  the  roles  of  champions  and  anti-champions  in  this  process? 

Do  you  have  a  policy  with  respect  to  use  of  office  computer  technology?  Please  describe 

9.  What  is  the  nature  of  relationships  with  outside  contractors? 

How  does  this  relationship  impact  technology  use? 

10.  Obtain  an  organizational  chart 


Interview  Protocol  11:  II:  Division  Chief  or  Branch  Manager  (or  Deputy) 

1.  Self  biography: 

Job  description/current  assignment 

Function/discipline 

Years  in  current  assignment 

Years  in  AF 

Years  on  base 

Education/degrees 

Computer  training 

2.  Describe  the  nature  of  your  group's  work  assignment. 

How  does  your  group  fit  witlW  the  SPO?  (relationship  to  SPO  mission;  matrix  relations) 

3 .  Does  your  group  deal  with  any  classified  (or  S  AR)  data? 

If  so,  what  percentage  of  the  work  is  secure? 

4.  Diagram  the  flow  of  work  to  your  group,  through  the  group,  and  out  of  your  group. 

Depict  other  groups  involved  in  the  flow. 

5.  In  what  stage  of  the  product  development  cycle  does  your  group  work  most  intensely? 

6.  Who  are  your  customers? 

How  is  information  shared  with  your  customers? 

7.  How  many  people  are  in  your  group? 

Describe  the  way  in  which  they  are  organized  (teams,  subtasks,  responsibilities). 
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Are  your  people  matrixed?  Describe  the  matrix  relations. 

Are  job  tasks  shared  or  individualized? 

What  are  the  disciplines  represented  in  your  group? 

What  other  groups  does  your  group  work  with  on  a  regular  basis? 

Are  there  any  perceived  differences  in  status  or  prestige  between  the  disciplines  represented  in 
your  group,  or  between  other  disciplines?  Describe. 

8.  Describe  the  nature  of  communications,  and  relationships  within  your  group. 

(Probe:  social  relations;  gaps  in  communication  flow  within  group,  between  groups,  between 
managers  and  workers) 

9.  What  is  the  best  division/branch  to  work  for  in  this  SPO? 

How  would  you  compare  your  division/branch  to  that  one? 

Is  there  another  division/branch  that  has  an  easier  time  recruiting? 

1 0.  What  is  the  size  of  the  budget  for  your  group? 

Has  the  budget  changed  in  the  last  two  to  three  years?  If  so,  how? 

1 1 .  How  many  contractors  does  your  group  work  with? 

What  other  off-base  organizations  does  your  group  work  with? 

What  kinds  of  data  is  exchanged  with  the  above?  How  is  it  exchanged? 

Describe  the  nature  of  the  relationships  you  have  with  the  above. 

Probe:  social  relationship. 

12.  Where  are  your  people  located?  (Numbers  and  locations) 

What  are  the  differences  in  resources  between  these  locations? 

Describe  the  interaction  between  people  located  in  different  places  (Communication) 

13.  What  is  the  nature  of  the  project-related  data  your  group  is  responsible  for?  Where,  and  how,  is 

this  information  stored? 

14.  Describe  any  changes  in  organizational  structure,  physical  location,  leadership,  or  personnel 

experienced  by  your  group  over  the  past  two  years. 

15.  What  kinds  of  office  computer  technology  do  you  use? 

What  kinds  of  office  computer  technology  are  used  by  your  group? 

(Probe:  E-mail,  shared  databases,  workflow  tools,  EDI,  imaging,  VTC, 

CAD/CAM) 

Describe  any  changes  in  the  office  computer  systems  used  over  the  past  two  years. 

Describe  the  group's  history  with  respect  to  computer  usage, 

(Probe:  evolution  of  usage,  barriers  and  facilitators) 

What  are  the  major  issues  involved  in  the  use  of  computers  within  this  group? 

(Probe:  contractor  issues,  security  issues) 
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16.  What  support  has  been  provided  to  enable  people  to  work  Avith  office  computer  technology? 
Describe  the  process  of  implementation  of  office  computer  technology.  Who  is  involved? 

Are  there  champions  or  anti-champions  for  office  computer  technology  use  in  your  group? 

Is  there  a  clear  understanding  of,  or  consensus  on,  the  need  for  and  goals  of  office  technology 
change? 


Interview  Protocol  HI:  Non-Management  Personnel 

1.  Self  biography; 

Job  description/current  assignment 

Function/discipline 
Years  in  current  assignment 
Years  in  AF 
Years  on  base 
Education/degrees 
Computer  training 

2.  Describe  your  current  job  assignment. 

What  kinds  of  office  computer  technology  tools  do  you  use  to  do  your  Job? 

(Probe.  E-mail,  shared  databases,  workflow  tools,  EDI,  imaging,  VTC,  CAD/CAM) 

3.  Who  do  you  work  with  most  closely  (consult  with  most  closely  or  most  of  the  time)? 

How  often  do  you  interact  with  these  people? 

Where  are  they  located? 

4.  Who  do  you  consider  your  group/team? 

What  is  the  size  of  this  group? 

How  do  people  work  together  in  your  group/team  to  accomplish  their  work? 

Are  job  tasks  shared  or  individualized? 

Are  you  matrixed? 

5.  Describe  the  nature  of  communications  and  relationships  within  your  group. 

(Probe:  social  relationships,  work  relationships,  communication) 

6.  Where  does  your  work  come  from  before  it  reaches  your  group? 

What  are  the  steps  the  work  goes  through  within  your  group? 

What  role  does  the  supervisor  play  in  the  work  flow? 

Where  does  the  work  go  after  it  leaves  your  group? 
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Diagram  the  work  flow. 

7.  What  other  groups  does  your  group  work  with  to  accomplish  its  tasks?  (Include  contractors) 

What  is  the  nature  of  the  interaction  with  these  groups? 

(Probe;  social  relationships,  work  relationships,  communication  -  gaps  between  workers, 
workers  and  managers?) 

8.  Describe  any  changes  in  organizational  structure,  physical  location,  leadership,  or  personnel 

experienced  by  your  group  over  the  past  two  years. 

9.  What  office  computer  tools  does  your  group  use  to  accomplish  its  work? 

(Probe:  E-mail,  shared  databases,  workflow  tools,  EDI,  imaging,  VTC,  CAD/CAM) 

Who  uses  the  tools/who  does  not  use  them'’  (Probe,  attitudes,  management  receptivity) 

To  what  extent  are  they  used? 

How  regularly  are  they  used'’ 

How  are  these  used  to  enable  or  support  the  relationships  with  other  groups? 

What  are  the  issues  or  concerns  that  have  arisen  regarding  the  use  of  computer  tools  to 
support  work? 

1 0.  What  are  the  advantages  and/or  disadvantages  of  existing  office  computer  technologies? 

1 1 .  Have  there  been  any  changes  in  office  computer  tools  in  the  last  year?  Describe. 

How  did  this  change  affect  the  work  of  the  group? 

What  issues  or  concerns  have  arisen  regarding  technology  change? 

12.  Are  there  any  plans  for  bringing  in  new  office  computer  tools?  Describe. 

How  will  the  new  technology  affect  your  work? 

What  issues  or  concerns  have  arisen  regarding  the  planned  changes? 

1 3 .  Describe  the  process  of  new  technology  implementation. 

Who  is  involved?  (Probe:  user  input,  computer  support  knowledge  and  abilities) 

What  are  the  steps  that  have  been  taken  in  the  past? 

What  actions  have  been  taken  to  prepare  for  the  arrival  of  new  office  computer  tools? 

Describe  the  training  given. 

14.  Are  there  champions  and/or  anti-champions  that  help  or  inhibit  the  office  technology  change 

process?  (Probe:  how  many,  who) 

Interview  Protocol  I\’;  Rapid  Assessment  Focus  Groups 
1 .  Self  biographies  (for  each  person  present): 
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Job  description/current  assignment 
Function/discipline 
Education/degrees 
Computer  training 

2.  What  is  the  nature  of  the  work  your  group  performs? 

Where  does  your  work  come  from,  what  do  you  do  with  it,  where  does  it  go? 

3.  How  do  you  work  together  within  the  group? 

(Probe:  social  and  work  relationships) 

Are  work  tasks  individualized  or  shared? 

Are  group  members  formed  into  teams?  If  so,  how  are  they  structured? 

How  do  you  communicate  within  the  group?  Why? 

4.  What  other  groups  inside  the  SPO  do  you  communicate  with? 

(Probe:  social  and  work  relationships) 

How  do  you  communicate? 

(Probe:  gaps  within  group,  between  groups,  between  managers  and  employees) 

5.  What  groups  outside  the  SPO  do  you  communicate  with? 

(Probe:  contractors) 

How  do  you  communicate  with  them? 

How  is  technology  used  in  this  relationship? 

What  have  been  the  barriers/facilitators  to  use  of  technology  with  these  outside  groups? 

What  is  the  quality  of  relationships  with  contractors? 

6.  Describe  the  amount  of  data  your  group  deals  with. 

Where,  and  how,  is  this  information  stored? 

7.  What  computer  tools  do  you  use  to  do  your  jobs? 

(Probe:  E-mail,  shared  databases,  worMow  tools,  EDI,  imaging,  VTC,  CAD/CAM) 

Who  uses  these  tools?  Who  does  not  use  them? 

To  what  extent  are  they  used? 

How  regularly  are  they  used? 

How  are  these  used  to  enable  or  support  the  relationships  with  other  groups? 

How  have  these  tools  impacted  your  work  process?  (Probe:  changes  in  work  process) 

What  are  the  issues  or  concerns  that  have  arisen  regarding  the  use  of  computer  tools  to  support 
work? 

Are  there  any  tools  that  you  are  missing?  Describe.  Why  are  they  missing? 

4 

8.  What  are  the  advantages/disadvantages  of  the  office  computer  tools  you  use? 

9.  What  is  the  history  oftechnology  implementation  and  usage  in  this  group? 
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What  changes  in  office  technology  have  occurred  in  the  past  two  years? 

Describe  the  process  of  new  technology  implementation. 

Who  is  involved’’  (Probe:  user  input,  computer  support  knowledge  and  abilities) 

What  are  the  steps  that  have  been  taken  in  the  past? 

What  actions  have  been  taken  to  prepare  for  the  arrival  of  new  office  computer  tools’’ 

Describe  the  training  given. 

What  are  the  barriers  to  technology  implementation/use? 

10.  Are  there  plans  for  new  office  technology  implementation’’ 

How  will  the  new  technology  affect  your  work? 

What  issues  or  concerns  have  arisen  regarding  the  planned  changes’’ 

1 1 .  Are  there  champions  and/or  anti-champions  that  help  or  inhibit  the  office  technology  change 

process’’  (Probe:  how  many,  who) 

12.  What  changes  has  this  group  experienced  over  the  past  two  years  in  organizational  structure, 
leadership,  personnel,  or  location? 

He************************************************************** 


Interview  Protocol  V:  Computer  Support 

1.  Self  biography: 

Job  description/current  assignment 

Function/discipline 

Years  in  current  assignment 

Years  in  AF 

Years  on  base 

Education/degrees 

Computer  training 

2.  Describe  your  current  job  assignment. 

3.  WTiat  groups  does  your  group  support’’  What  is  the  nature  of  the  interaction  with  these  groups? 

(Probe:  social  relationships,  work  relationships,  communication  -  gaps  between 
computer  support  and  workers,  computer  support  and  management) 

4.  Describe  any  changes  in  organizational  structure,  physical  location,  leadership,  or  personnel 

experienced  by  your  group  over  the  past  two  years. 

5.  What  office  computer  technologies  are  used  by  the  people  you  support? 

(Probe:  E-mail,  shared  databases,  workflow  tools,  EDI,  imaging,  VTC,  CAD/CAM) 
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6.  What  are  the  advantages  and/or  disadvantages  of  existing  office  computer  technologies? 

7.  Have  there  been  any  changes  in  office  computer  tools  in  the  last  year?  Describe. 

What  issues  or  concerns  have  arisen  regarding  technology  change? 

8.  Are  there  any  plans  for  bringing  in  new  office  computer  tools?  Describe. 

What  issues  or  concerns  have  arisen  regarding  the  planned  changes? 

9.  Describe  the  process  of  new  technology  implementation. 

Who  is  involved?  (Probe:  user  input,  computer  support  knowledge  and  abilities) 

What  are  the  steps  that  have  been  taken  in  the  past? 

What  actions  are  taken  to  prepare  for  the  arrival  of  new  office  computer  tools? 

Describe  the  training  given. 

10.  Are  there  champions  and/or  anti-champions  that  help  or  inhibit  the  office  technology  change 

process?  (Probe:  how  many,  who) 

11.  What  are  the  attitudes  of  the  people  in  the  groups  you  support,  with  respect  to  office  computer 

technology? 
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Appendix  C:  Software  Design 


This  appendix  outlines  the  software  development  part  of  the  FRAMEAVORK  project, 
FRAME, AVORK  software  is  a  cultural  assessment  tool  researched  and  developed  by  Wizdom 
Systems,  Inc.  in  Naperville,  IL  under  a  Phase  II  SBlR  contract  #  FA1624-93-C-5016.  This  tool 
assesses  the  obstacles  in  deploying  new  technologies  in  the  Air  Force  and  recommends  some 
solutions. 


C.l  Requirements 


The  research  team,  headed  by  Dr.  Allen  W.  Batteau,  came  up  with  the  requirements  including: 

1 .  be  able  to  run  under  Microsoft  Windows  environment, 

2.  be  able  to  display  questions  to  the  users  and  store  the  answers,  and 

3.  be  able  to  draw  conclusions  based  on  the  answers  given  by  the  users  and  to  offer 

recommendations. 

The  first  requirement  asks  for  the  development  done  under  the  Windows  environment.  We 
decided  to  use  a  combination  of  Visual  Basic  and  Clips.  The  justification  is  discussed  under 
section  C.4. 

The  second  requirement  asks  the  tool  to  have  the  ability  to  retrieve  questions,  display  them  to  the 
users,  and  log  in  users  answers.  This  process  could  be  done  in  a  number  of  ways.  We  chose  to  use 
a  database.  Databases  support  structured  data  storage  and  retrieval  and  have  many  options  to 
choose  from.  Databases  are  widely  used  by  almost  all  the  organizations  to  store  and  retrieve 
structured  data. 

The  third  requirement  called  for  an  expert  system.  An  expert  system  is  able  to  store  the  human 
knowledge  in  its  knowledge  base,  to  use  it  to  against  some  facts,  and  to  draw  conclusions.  There 
are  some  alternatives  to  this  approach,  but  none  of  them  offers  the  modularity,  maintainability, 
and  ease  of  use  offered  by  the  expert  systems. 


C.2  Design 
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Based  on  the  requiremf^nts,  we  decided  to  divide  the  main  program  into  several  modules  as 
depicted  in  Figure  C 1 . 


Figure  Cl  -  Software  Modules  for  FWTOOLl.EXE 


1 .  The  User  Interface  Module 

The  user  interface  module  is  responsible  to  display  questions  on  the  screen,  handle  menu 
selections,  get  user  choices,  and  perform  user  requests.  The  design  of  the  user  interface  module 
follows  the  standard  Windows  visual  design  conventions  to  minimize  the  users  leam  curve.  For 
example,  the  left  most  top  menu  is  the  file  menu  and  the  right  most  one  is  the  help  menu,  Since 
the  tool  does  not  have  editing  capability,  the  edit  on  menu  was  not  implemented.  Instead,  a  view 
menu,  which  allows  the  users  to  navigate  through  the  questions,  and  an  options  menu,  which 
allows  the  users  to  change  the  font  and  color  of  the  display,  were  implemented.  The  justification 
for  the  view  menu  design  is  obvious  because  we  have  to  allow  the  users  to  look  at  previous 
questions  at  any  time  so  they  can  edit  the  questions  or  refresh  what  they  have  just  answered.  The 
option  menu  is  an  added  feature  for  users’  convenience. 

The  questions  of  the  assessments  are  divided  into  four  groups:  commander,  subdivision  officer, 
computer  specialist,  and  end  user.  To  distinguish  the  questions  so  that  a  user  knows  which  group 
of  questions  he/she  is  answering,  one  of  the  four  different  frames  is  drawn  on  the  screen  based  on 
which  group  of  questions  is  displayed.  When  the  tool  asks  the  user  which  group  of  questions 
he/she  is  going  to  answer,  the  same  four  frames  are  drawn  around  four  pictures  which  represent 
the  four  different  groups.  This  visual  linkage  reminds  the  users  which  group  of  questions  he/she  is 
answering. 
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Every  effort  had  been  made  to  ease  the  guessing  game  of  how  to  answer  a  question.  If  radio 
buttons  are  displayed,  then  the  users  know  only  one  selection  is  possible.  If  check  boxes  are 
displayed,  however,  many  choices  may  be  selected.  A  spinner  indicates  a  numeric  answer  is 
expected,  and  a  text  box  allows  the  users  to  type  any  needed  information. 


2.  The  Database  Interface  Module 

The  database  interface  module  handles  the  retrieval  questions  from  and  storage  of  answers  to  the 
database.  When  the  user  interface  module  needs  to  display  the  next  question,  it  calls  the  next 
question  function  in  this  module  to  retrieve  it.  This  function,  in  turn,  gets  the  question  from  the 
database  and  passes  it  to  the  user  interface  module.  When  the  users  answer  the  question  and  are 
ready  to  move  to  the  next  one,  the  user  interface  module  sends  the  answer(s)  to  the  record  answer 
function  in  this  module.  This  function  passes  the  answer(s)  to  the  database.  Other  fimctions  in  this 
module  work  in  the  similar  manner. 

This  design  hides  the  database  implementation  details  from  the  user  interface  module.  The  module 
provides  a  number  of  functions,  (e  g.  next  question),  for  the  user  interface  to  call.  The  user 
interface  does  not  know  how  and  where  to  get  the  questions.  In  this  way,  any  changes  made  in  the 
database  module  is  hidden  so  the  tool  is  easily  maintained. 

Using  the  standard  database  design  principle,  the  assessment  database  has  six  tables: 

1 .  The  question  table:  This  table  has  all  the  questions. 

2.  The  choice  table:  This  table  has  all  the  choices  for  the  questions. 

3.  The  assessment  table:  This  table  has  general  information  about  each  assessment. 

4.  The  responder  table:  This  table  records  the  responders  information  of  the  assessments, 
5  The  assessment  responses  table:  This  table  records  all  the  answers  to  the  assessments. 

6.  The  text  object  table:  this  table  has  long  text  v/hich  cannot  be  held  by  other  tables. 


3.  The  Expert  System  Module 

Like  the  database  module,  the  expert  system  module  handles  the  interface  between  the  main 
program  (fwtools.exe)  and  the  external  expert  system.  This  module  offers  two  functions  to  the 
user  interface  module.  When  a  user  wants  to  see  the  recommendations,  the  interface  module  calls 
the  consult  expert  function  to  start  the  reasoning  process.  When  the  reasoning  process  ends,  a 
number  of  issues  are  displayed  to  the  users.  If  the  user  selects  one  of  the  issues,  the  selected  issue 
is  displayed  on  the  screen  by  the  look  up  issue  function.  This  design  offers  the  same  advantages 
discussed  in  the  database  module. 
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4.  The  Print  Module 


This  module  was  not  on  the  initial  design  and  was  added  later.  The  request  of  being  able  to  print 
the  assessment  in  a  particular  format  called  for  the  creation  of  this  module  because  printing  should 
not  be  the  responsibility  of  the  above  three  modules.  This  module  exports  the  print  assessment 
fiinction  to  the  user  interface  module  so  that  when  this  function  is  called,  the  questions  and 
answers  from  the  database  are  retrieved,  formatted,  and  printed  on  the  default  printer. 


C.3  Implementation 


The  implementation  of  the  tool  was  dictated  by  several  factors  . 

1 .  This  is  a  prototype  of  the  would-be  commercial  assessment  tool. 

2.  The  tool  had  to  be  produced  very  fast  so  that  the  research  team  could  use  it  to  gather 
more  data  in  the  field. 

3.  There  was  limited  man  power  in  the  development  team.  The  author  of  this  section  of 
the  report  was  the  only  full  time  developer.  There  was  another  engineer  in  Wizdom 
Systems,  Inc.  who  worked  only  part  time  initially. 

It  is  obvious  we  needed  a  fast  development  tool  to  accomplish  the  design,  given  the  limitations. 
Visual  Basic  was  selected  to  implement  the  user  interface  module  because  it  offers  a  fast  user 
interface  development  and  has  the  built-in  database  connections.  These  two  features  made  Visual 
Basic  a  better  selection  than  other  software  programs,  (e  g.  C++  implementation).  Visual  Basic  is 
slower  than  other  implementations  because  it  is  an  interpreted  language.  Yet  the  program 
performance  is  acceptable  partially  because  users  spend  the  most  time  reading  questions  and 
thinking  about  answers,  and  partially  because  many  performance  optimization  techniques  were 
applied  in  the  code. 

Although  Visual  Basic  can  handle  the  requirements  for  the  main  program,  it  has  no  way  to 
implement  the  expert  system.  We  wanted  an  expert  system  be  able  to  run  under  Windows  and  be 
transparent  to  the  users,  preferably  a  Windows  DLL  program.  Some  systems,  like  GURU,  do  run 
under  Windows  as  EXE  program  with  a  high  price.  Some  other  systems  do  not  run  under 
Windows  at  all.  We  finally  chose  CLIPS  (C  Language  Implementation  of  Production  Systems) 
developed  by  NASA  Johnson  Space  Center.  Not  only  does  it  run  under  Windows  (version  6.0), 
but  also  it  gives  out  the  source  code  in  C.  None  of  the  other  expert  systems  have  these  two 
advantages  combined.  It  was  also  reasonably  priced. 


1 .  User  Interface  Implementation 
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Given  the  design,  the  implementation  of  the  user  interface  is  straightforward.  There  were, 
undoubtedly,  many  technical  problems  to  overcome.  One  of  the  problems  was  the  performance. 
When  we  implemented  question  display  and  clear  in  a  standard  way,  the  performance  was  poor. 
The  screen  was  painted  too  slow  and  the  background  frame  had  to  be  repainted  each  time  a  new 
question  was  displayed,  which  caused  flicker  on  the  screen.  We  tried  many  solutions  and  ended  up 
with  the  solution  to  repaint  the  screen  with  the  same  question  text  (but  background  color)  when 
we  wanted  to  clear  the  screen.  This  solution  worked  very  fast  and  no  flickers  occurred. 


2.  Database  Interface  Implementation 

One  of  the  reasons  to  choose  Visual  Basic  iss  that  it  has  built-in  database  support.  In  Visual 
Basic,  one  can  create  databases,  create  tables,  insert  data,  retrieve  data,  and  perform  SQLs.  For 
this  project,  we  only  need  to  retrieve  data  from  and  store  data  to  the  database. 

Visual  Basic  provides  three  ways  to  open  a  database  table;  as  table,  as  dynaset,  or  as  snapshot. 
Snapshot  is  “read-only,”  so  we  used  it  for  the  question  and  choice  tables.  For  the  other  four 
tables,  we  could  use  either  the  table  objects  or  the  dynaset  objects  to  open  them.  We  chose  to  use 
the  dynaset  method  because  it  supports  SQL,  which  we  needed  in  some  database  operations. 

We  used  dBASE  IV  for  our  database  implementation  simply  because  our  products,  like 
ProcessWorks!,  use  the  same  database.  We  could  use  other  databases  as  well.  Since  Client/Server 
was  not  a  requirement,  we  simply  used  local  databases. 


Expert  System  Interface 
Module 

Inference  Engine 

Knowledge  Rae 


Figure  C2  -  External  Expert  System 


3.  Expert  System  Interface  Implementation 
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The  external  expert  system  has  two  components  as  depicted  in  Figure  2.  The  inference  engine  is 
the  brain  of  the  expert  system  and  it  does  all  the  reasoning  processes.  The  inference  engine  uses 
the  rules  stored  in  the  knowledge  base  and  the  facts  (answers)  from  the  main  program  to  draw 
conclusions.  The  knowledge  base  is  a  text  file  which  stores  all  the  rules  supplied  by  the  research 
team  including  some  initial  facts. 

We  wanted  the  inference  engine  to  run  as  a  Windows  DLL  program  but  CLIPS  does  not  have  it. 
So  we  took  the  source  code,  which  was  written  in  C,  and  converted  and  compiled  it  as  a 
Windows  DLL.  One  aspect  of  this  conversion  was  the  size  of  the  DLL.  Initially,  we  compiled  the 
code  into  DLL  with  most  features  turned  on.  The  size  of  the  program  was  900K.  We  thought  that 
was  too  big.  So  we  turned  off  many  unnecessary  features  and  the  size  of  the  released 
FWEXPERT.DLL  was  dropped  to  200K. 

The  knowledge  base  implementation  is  dictated  by  the  choice  of  the  expert  system.  Since  we 
chose  CLIPS,  we  must  use  CLIPS  syntax  to  write  our  rules.  Even  so,  we  still  have  choices  for  the 
reasoning  models.  Based  on  the  nature  of  this  project,  the  author  of  this  report  proposed, 
designed,  and  implemented  a  Conditional  Probability  reasoning  model  in  the  FRAMEAVORK 
tool.  The  Conditional  Probability  model  takes  care  of  the  uncertainty  of  the  problem  by  adjusting 
the  probability  of  each  issue,  based  on  each  answer  to  the  questions.  Please  refer  to  the 
knowledge  base  file,  FWCOND.CLP,  for  more  details. 


C.4  Further  development  opportunities 

The  design  and  implementation  of  FRAMEAVORK  were  successful.  The  program  satisfied  all  the 
requirements  and  is  easily  maintained.  During  the  course  of  design  and  implementation,  the 
requirements  had  been  changed  many  times  and  we  were  able  to  accommodate  these  changes  in 
time  because  of  the  initial  module  design. 

The  FRAMEAVORK  tool  was  designed  to  be  a  prototype  program  so  there  is  a  process  to 
convert  it  to  a  commercial  product.  We  believe  the  following  steps  are  necessary  to  do  the 
conversion: 

1 .  Restructure  the  database:  some  fields  on  some  tables  may  be  eliminated  because  they 
are  no  longer  needed.  Change  the  database  to  Microsoft  Access  to  improve 
performance. 

2.  Modify  the  inference  engine:  current  inference  engine  can  only  take  rules  as  text  files. 
This  process  is  slow,  and  proprietary  information  in  the  knowledge  base  cannot  be 
protected.  Converting  the  inference  engine  to  a  run-time  version  solves  both  problems. 
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Appendix  D:  Software  Documentation 


USAF  FRAME/ WORK  Readiness  Assessment  Tool 
Developed  under  contract  with 
Wizdom  Systems,  Inc. 


User  Guide 


The  USAF  FRAMEAVORK  Readiness  Assessment  Tool  is  an  assessment  and  expert  system  for 
evaluating  the  human  issues  in  information  technology  implementation.  It  is  intended  to  guide 
management  personnel  in  anticipating  user  requirements  and  issues  as  they  implement  one  of  the 
following  types  of  systems; 

e-mail 

workflow  tools  (e.g. ,  Lotus  Notes) 

engineering  and  logistics  databases  (e  g.,  JEDMICS) 

video  teleconferencing 

CAD/CAM 

document  imaging  (electronic  storage  &  retrieval) 
electronic  data  interchange  (X.  12-compliant  systems) 

The  tool  consists  of  three  parts:  an  assessment  questionnaire,  an  expert  system  for  analyzing  the  results 
from  the  questionnaire,  and  a  hypertext-based  recommendation  system.  The  recommendation  system 
presents  the  issues  that  are  highlighted  by  the  questionnaire,  and  a  set  of  recommendations. 

FRAMEAVORK  is  a  top-down  tool.  It  is  intended  to  assist  the  leadership  of  an  organization,  and  is 
structured  so  that  the  command  must  initiate  its  use. 

These  issues  and  recommendations  are  based  on  research  conducted  in  AFMC  System  Program 
Ofiices  fi'om  October  1992  to  October  1994.  The  research  was  conducted  by  a  team  consisting  of 
personnel  fi'om  Wizdom  Systems,  Inc.,  and  Wayne  State  University. 

Additional  information  on  the  development  of  FRAMEAVORK  can  be  obtained  fi^om  Wizdom 
Systems,  Inc. 
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1.  General  Concepts 


FRAMEAVORK  is  a  readiness  assessment  and  management  support  tool  for  the  task  of  implementing 
new  systems  in  a  program  environment.  There  are  three  core  concepts  that  are  embodied  in  the  tool: 

•  Human  issues  facilitate  or  impede  technology  implementation.  The  use  of 

advanced  systems,  particularly  those  that  change  peoples’  jobs  (such  as  EDI  or 
workflow  tools),  is  sometimes  furthered,  and  sometimes  hindered  by  issues  such  as 
cultural  attitudes,  organizational  barriers,  or  a  command's  previous  history  with 
implementation.  If  these  issues  are  not  managed,  the  implementation  may  fail. 

•  The  solution  to  these  issues  must  be  managed,  not  engineered.  If  a  manager  is 
aware  of  the  issues,  he  or  she  will  have  the  resources  necessary  to  address  them. 
However,  managing  the  issues  is  dependent  on  numerous,  related  contextual  issues 
which  only  the  manager  can  be  aware  of 

•  A  tool  is  no  substitute  for  management.  A  tool  such  as  FRAMEAVORK  can  collect 
data,  analyze  data,  alert  management  to  issues,  focus  the  issues  for  the  specific  context, 
and  provide  recommendations.  However,  acting  on  the  issues  requires  leadership  on 
the  part  of  the  manager. 

Within  these  boundaries,  the  FRAMEAVORK  team  has  developed  a  tool  and  a  knowledge  base  that 
accurately  reflects  Air  Force  issues,  and  places  this  knowledge  in  the  hands  of  the  manager. 


2.  Tool  Architecture 

The  FRAMEAVORK  tool  consists  of  five  components,  all  of  which  run  under  a  common  Microsoft 
Windows  3 . 1  graphical  user  interface.  The  five  components  are: 

1.  Session  and  assessment  manager.  This  is  a  series  of  six  questions  for  setting  up  an 
assessment  and  establishing  the  nature  of  the  session  and  user.  When  a  new  assessment 
is  created,  it  elicits  the  command's  viewpoint  on  technology  issues,  and  what  type  of 
system  is  being  implemented. 
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2.  Assessment  module.  A  series  of  questions  (some  for  the  command,  some  for 
managers,  some  for  IT  support  personnel,  and  some  for  users)  to  assess  the 
implementation  environment. 

3  Expert  system/inference  engine.  A  knowledge  processor  that  takes  the  results  of  the 
assessment,  and  identifies  the  salient  issues  in  the  organization. 

4.  Hypertext  recommendation  module.  As  issues  are  identified,  the  user  sees  screens 
describing  them,  and  can  click  on  different  parts  of  the  definition  for  recommended 
actions  for  the  issue. 

5,  Session  log  and  database.  As  assessment  information  is  put  into  the  tool,  it  is 
recorded  in  a  database.  The  tool  can  store  up  to  five  assessments,  each  codenamed 
■with  the  name  of  a  famous  aircraft.  Storage  of  this  information  permits  the  data  to  be 
analyzed  at  any  time  by  any  version  of  the  expert  system. 

These  five  components  form  a  loosely  coupled  system,  permitting  maximum  flexibility  to  both  the  users 
and  the  developers. 


3.  System  Requirements 


FRAME/WORK  is  designed  to  run  under  Microsoft  Windows  3.1  on  a  80386-  or  80486-based 
platform.  It  requires  VGA  graphics  or  higher,  and  a  mouse.  It  requires  a  3,5”  floppy  drive  for 
installation,  and  at  least  5  mb  disk  space  on  the  hard  drive.  We  recommend  at  least  a  25mHz  processor 
for  best  system  performance. 

One  section  of  the  assessment  must  be  completed  by  at  least  six  system  users.  The  users  can  either 
provide  their  input  directly  into  the  tool,  or  can  complete  a  paper  questionnaire.  If  the  latter  is  desired, 
the  system  will  generate  the  paper  questionnaire.  This  paper  questionnaire  can  then  be  returned  for 
input  into  the  FRAME/WORK  tool. 


4.  Installation 


To  install  the  FRAMEAVORK  tool,  simply  insert  the  first  installation  disk  into  the  floppy  drive  of  your 
computer  While  in  Windows,  run  the  file  manager.  Double  click  on  the  A:  drive  (if  A:  is  the  3.5" 
drive  on  your  computer).  You  will  see  a  directory  listing  for  the  A:  drive. 
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Scroll  down  (if  necessary)  until  you  see  a  file  named 

SETUP.EXE 

Double-click  on  the  icon  for  this  file.  The  setup  routine  will  run  automatically. 


5.  User  Guide 


Configuration 

The  configuration  cycle  consists  of  a  few  questions  designed  to  set  up  the  assessment  (if  you  are 
creating  a  new  one),  or  establish  your  role  with  regard  to  an  existing  assessment.  The  questions  are: 

Q  1(5);  In  this  assessment,  are  you 

_ Managing  the  assessment  (create  new  assessment,  re-use 

previous  assessment  for  a  new  implementation,  delete  assessment). 

_ Providing  input  to  the  assessment. 

_ Examining  the  results  (review  responses,  examine  issues). 

_ Doing  none  of  the  above;  quit  the  assessment 

If  there  are  no  active  assessments,  only  the  first  and  last  options  will  be  highlighted. 


Q  2(7):  Which  of  the  following  active  assessments  do  you  wish  to 
provide  input  for  or  review? 

_ Flying  Fortress 

_ Tomcat 

_ Mustang 

_ Liberator 

_ Sabrejet 

Only  those  that  are  active  will  be  highlighted.  You  should  act  on  this  question  promptly,  as  the  tool 
will  otherwise  select  the  first  on  the  list. 


Q  3(10):  Do  you  wish  to 
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_ Create  a  new  assessment 

_ Re-use  a  previous  assessment 

_ Delete  an  assessment 

This  question  is  available  to  the  assessment  manager  only. 


Q  4(  1 5):  Please  select  a  name  from  the  following  list  for  this  assessment.  This  name  will  be  used 
to  identify  the  assessment  for  all  subsequent  input  and  review: 

_ Flying  Fortress 

_ Tomcat 

_ Mustang 

_ Liberator 

_ Sabrejet 

Assessments  are  given  code  names  from  this  list. 


Q  5(20):  Is  this  tool  being  used  to  examine  issues  at  the  level  of  a 

_ Program  office  or  directorate  (2-letter) 

___  Division  (3-letter) 

_ Branch  (4-letter) 

Depending  on  which  organizational  level  is  being  examined,  different  questions  will  be  asked. 

Q  6(30):  What  type  of  information  technology  are  you  planning  to  implement  at  this  time? 

_ E-mail  (e  g.,  PROFS) 

_ Decision  support  databases 

_ Engineering  data  databases  (e  g.,  EDMICS) 

_ CAD  or  CAM  systems  (e  g..  Unigraphics,  CadKey,  Mentor, 

AutoCad) 

_ Electronic  Data  Interchange  (X.  12  compliant  systems) 

_ Document  Imaging 

_ Video  Teleconferencing 

_ Workflow  tools  (e  g.,  Lotus  Notes) 

_ Logistics  databases 

Q  7(32):  Please  select  from  this  list  of  active  assessments  the 
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one  that  you  wish  to  use. 

_ Flying  Fortress 

_ Tomcat 

_ Mustang 

_ Liberator 

_ Sabrqet 

Q  8(34);  Please  select  from  the  following  list  a  name  that  you 
wish  to  identify  this  new  assessment  by: 

_ Flying  Fortress 

_ Tomcat 

_ Mustang 

_ Liberator 

_ Sabrejet 

Q  9(36);  Please  select  from  this  list  of  active  assessments  the 
one  that  you  wish  to  delete. 

_ Flying  Fortress 

_ Tomcat 

_ Mustang 

_ Liberator 

_ Sabrejet 

Q  10(37):  Please  click  on  the  box  that  indicates  the  type  of  input  you  are  providing 
_ User 

_ Organizational  Subdivision 

_ Computer  Specialist 

_ Overall  Organization 

Q  1 1(38):  Are  you 

_ Providing  new  input 

_ Editing  previous  input  or  resuming  a  previous  session 

_ Deleting  previous  input 

Q  12(39):  Do  you  wish  to 

_ Review  the  input  of  this  assessment 

_ Examine  and  respond  to  the  issues 
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_ See  the  recommendations  of  a  previously  completed  assessment 

Q  1 3(40):  Which  of  the  following  is  the  objective  of  this 
implementation?  (Check  all  that  apply.) 

_ Improve  productivity 

_ Improve  communication  with  remote  co-workers  (outside  of 

the  building) 

_ Improve  communication  with  other  sites  on  base 

_ Improve  communication  within  work  groups  (within  the 

building) 

_ Streamline  workprocesses 

_ Improve  the  quality  of  our  work 

_ Other; 

Don't  know 


These  questions  are  then  used  to  configure  the  session. 


Assessment 

The  Assessment  Cycle  consists  of  four  parts:  Command,  Manager,  System  Support,  and  User  After 
you  have  indicated  what  type  of  input  you  are  providing,  you  are  given  a  code  number,  this  is  for  you 
to  remember,  in  case  you  wish  to  review  or  revise  your  input,  or  provide  the  input  in  two  or  more 
separate  sessions 

The  Command  Assessment  consists  of  approximately  60  questions.  Depending  on  how  certain 
questions  are  answered,  some  questions  may  be  skipped  Completing  the  Command  Assessment 
requires  about  30  minutes. 

Most  of  the  questions  in  the  Command  Assessment  are  self-explanatory.  A  few  questions  deserve 
explanation  here: 

90  What  is  the  office  symbol  of  the  group  that  you  are  responsible  for? 

Enter  the  two-,  three-,  or  four-letter  symbol  of  your  directorate,  division,  or  branch. 


1 80  How  many  organizational  subdivisions  report  to  you'’ 
Enter  a  number. 
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185  Please  highlight  the  organizational  symbol  for  each  of  the  subdivisions  that  reports  to 
you. 

FRAMEAVORK  conducts  a  separate  assessment  of  impacted  subdivisions.  On  this  question, 
you  need  to  highlight  one  or  more  organizational  symbols.  The  tool  will  give  you  a  list  of 
organizational  symbols  based  on  the  response  to  the  previous  question,  you  highlight  these 
with  the  mouse,  and  then  add  the  subdivision  letter  (e.g.,  add  an  "E"  to  "AV"  to  make  "AVE", 
and  then  click  ADD) 


190  For  the  information  technology  you  are  implementing,  will  this  affect  all  of  these 
subdivisions?  If  not,  please  scroll  through  the  list  and  indicate  which  subdivisions  will  be 
affected. 

Flighlight  the  significant  subdivisions.  The  tool  will  then  generate  questions  for  each  of  the 
subdivisions  highlighted. 


Manager  Cycle 

The  Manager  Cycle  in  the  assessment  consists  of  approximately  48  questions.  Some  of  the  questions 
are  determined  by  what  sort  of  technology  is  being  implemented.  This  section  will  require 
approximately  30  minutes  for  each  manager  to  complete.  All  of  the  questions  are  self-explanatory. 
However,  this  section  should  be  completed  for  each  organizational  subdivision  affected  by  the 
implementation.  Thus,  if  in  AQ,  the  subdivisions  AQK,  AQC,  and  AQL  are  affected,  but  not  AQE, 
this  section  should  be  completed  for  the  first  three  only.  The  tool  will  ask  in  the  configuration  cycle 
what  subdivision  you  are  completing. 


System  Support  Cycle 

Certain  questions,  having  to  do  mostly  with  hardware  and  software,  will  probably  best  be  completed  by 
your  systems  support  person.  There  are  28  of  these,  requiring  approximately  20  minutes. 


User  Cycle 

The  great  power  of  this  tool  is  in  the  feedback  it  provides  to  the  command  on  user  issues.  For  this 
reason,  it  is  valuable  to  have  a  good  representation  of  users  providing  input.  This  input  c^  be 
provided  either  on-line,  or  else  by  printing  a  paper  copy  of  the  user  questionnaire  (click  on  "file", 
"print",  "user  questions"). 
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The  user  section  consists  of  approximately  65  questions,  requiring  30  minutes  per  user  to  complete. 
We  recommend  that  at  least  6  users  provide  input;  the  more  users,  the  more  complete  your  sample. 

All  of  the  questions  are  self-explanatory. 


Recommendations 

At  any  time  you  may  examine  the  recommendations  of  the  tool.  You  do  this  by  clicking  on 
"recommendations".  The  tool  will  then  provide  you  with  a  list  of  five  issues.  Click  on  one  of  these, 
and  it  will  provide  a  screen  describing  the  issue.  Certain  parts  of  the  screen  will  be  highlighted  in  green. 
By  clicking  on  a  green  highlight,  you  will  jump  to  a  screen  providing  a  recommendation  for  that  issue. 
Click  on  "back"  to  return  to  the  issue  or  to  the  main  recommendation  menu. 


6.  Further  information  and  technical  support 


For  further  information  on  the  FRAMEAVORK  tool,  contact; 

Mr.  Philip  Clement 
Wizdom  Systems,  Inc. 

1300  Iroquois  Avenue 
Naperville,  Illinois  60563 
708-357-3000;  fax  708-357-3059 
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Appendix  E:  SPO  Observations  and  Findings 


In  the  research  for  the  FRAMEAVORK  project,  nine  different  organizations  were  examined,  and 
detailed  ethnographic  data  were  collected  on  each.  Presented  here  are  two  summaries  of  these  data, 
the  levels  of  implementation  of  seven  different  types  of  systems  (e-mail,  shared  databases,  workflow 
tools,  video  teleconferencing,  document  imaging,  Epi,  and  CAD/CAM),  and  our  assessment  of  the 
organization  on  nine  critical  issues  (t5^e  of  organization,  mission,  size,  fragmentation,  turbulence, 
implementation  process,  assumptions  concerning  computing,  prestige,  and  security.  These  data  are 
presented  on  the  next  nine  pages. 
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SPO  #1  -  Working  in  the  Vault 


Application 

Systems  used 

Comment 

e-mail 

All-in-One;  AMS  e-mail;  base  e-mail; MIS 

not  in  work  area;  accessed  once  a  day, 

e-mail 

not  by  everyone 

shared  databases 

Threat  database;  Caretaker  II;  Federal 

Threat  database  and  Caretaker  II 

Acquisitions  Regulations;  AF 

disconnected  when  individual  user  left 

Acquisitions  Model 

the  org.;  FAR  outside  work  area;  AFAM 
used  by  one  person 

workflow  tools 

Automated  Management  System  (AMS); 

last  3  tool  used  by  single  individuals;  2nd 

project  management  software; 

and  3rd  tools  not  mentioned  during 

simulation  software;  cost  modeling 

interviewing  but  mentioned  by 

software;  performance  analyzer 
program;  configuration  management 
system 

management  in  outbriefing 

video  teleconferencing 

None 

document  imaging 

K5200 

not  used 

EDI 

White  Knight 

use  dicontinued  when  individual  user  left 
the  organization 

CAD/CAM 

None 

Systems  Profile,  SPO  #1 


Cultural  dimension 

Key  features 

Comment 

Type  of  organization 

Basket  dividsion  of  basket  SPO 

Mission 

Acquisition 

Size 

c.  60 

Fragmentation 

All  in  same  building  frequent  TDY 

2  vaults  on  different  floors  of  same 
building;  2  separate  sections  in  one 
vault 

Turbulence 

Eliminating  branches;  creating  IPT's;  additional  documentation  required; 
downsizing=personnel  loss;  new  leadership 

Implementation  process 

No  user  input;  history  of  failures;  people 

Computer  support  not  seen  as  champs 

not  informed  of  plans 

or  knowledgeable  in  usage  of  systems 

Assumptions  re  computing 

Unreceptive  culture 

Prestige 

Presitigous  due  to  security  [v] 

Security 

All  data=hlgh  classification  [v] 

Important  in  office  automation 

Cultural  Profile,  Organization  1 
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Component  #2  -  Keeping  them  flying 


Application 

Systems  used 

Comment 

e-mail 

Appletalk 

Used  only  by  some  engineers,  security, 
and  command  section 

shared  databases 

Mainframe  programs:  1)NSN:  2)part 
numbers,  shipping  and  receiving,  and 
inventory  systems;  3)PC3  personnel 
system 

Used  by  members  of  one  branch  only 

workflow  tools 

None 

video  teleconferencing 

None 

document  imaging 

None 

EDI 

None 

CAD/CAM 

None 

Systems  Profile,  Component  #2 


Cultural  dimension 

Key  features  Comment 

Type  of  organization 

Basket  dividsion  of  basket  SPO 

Mission 

Sustainment 

Size 

c.  400?? 

Fragmentation 

All  in  same  building  some  TDY  One  group  of  offices  separated  from  the 

rest 

Turbulence 

Move  under  new  command  combining  logistics  and  engineers;  additional 
documentation  required;  increased  reguiatory  oversight;  downsizing=personnel 

loss;  leadership  changes 

Implementation  process 

No  user  input;  history  offailures;  no  plans  Computer  support  not  seen  as  champs 

or  supporting  of  entire  org 

Assumptions  re  computing 

Resistive  culture 

Prestige 

Presitigous  due  to  security  [v] 

Security 

All  data=high  classification  [v] 

Cultural  Profile,  SPO#2 
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Component  #3  —  Planning  for  support 


Application 

Systems  used 

Comment 

e-mail 

LAN  E-mail 

shared  databases 

LSAR(Logs  Sppt  Analy  Rees); 

Most  are  databases  specific  to  logistics 

DAMIS(Depot  Activ  MIS);  TIS(Test  Info 

functions;  most  used  by  a  number  of 

Sheet);  STS(Suspense  Tracking  Sys); 
TAC(Technical  Acquisitions);  CLASS 
(Comprehensive  Logs  anal  Sppt  Sys); 
D-143;  220;  GO-57;  72D;  79;  81;  DO- 
32;  34;  35;  39;  41;  43;  62;  63;  87; 

G005M 

people  in  the  organization 

workflow  tools 

MIS  (Mgmt  Info  Sys) 

video  teleconferencing 

Available 

Used  extensively  by  mgmt 

document  imaging 

None 

EDI 

None 

CAD/CAM 

None 

Systems  Profile,  Component  #3 


Cultural  dimension 

Key  features 

Comment 

Type  of  organization 

Single  program  SPO 

Mission 

Sustainment 

Size 

c.  130 

Fragmentation 

All  in  same  building  infrequent  TDY 

Maintenace  crew  In  another  building 

Turbulence 

Move  under  new  command;  creation  of  IPTs;  increase  in  personnel,  but  decrease 

in  authorized  positions  available;  leadership  changes;  policy  changes 

Implementation  process 

No  user  input;  history  of  failures;  most 

Computer  support  not  seen  as  champs; 

people  informed  of  plans 

have  other  duties  &  not  located  in  same 
building 

Assumptions  re  computing 

Some  resistance  in  culture 

Prestige 

Low  prestige  due  to  Congress 

Security 

Little  classified  data 

Cultural  Profile,  Organization  3 
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SPO  #4  -  Global  mission 


Application 

Systems  used  Comment 

e-mail 

AINn-One 

shared  databases 

ASPIN;  AMS;  TICARS;  PMD;  MRP;  API;  POD;  API;  GIDEP;  USD;  and  the  PMD 
POD;  D043;  G021;  G023;  FAR;  ISA;  were  used  by  single  individuals;  5  used 
GIDEP;  FIN  (Lockheed);  FOXPRO;  by  people  in  each  branch;  1/3  used  by 

SERD;  SESATS;  PMA;  USD  more  than  one  person  in  each  branch 

workflow  tools 

Shared  Drives;  Cost  Calculator;  Job  Shared  Drives  used  by  a  number  of 

Descriptions  &  Codes;  Air  Force  Forms;  organization  personnel;  rest  used  only  by 
Document  Tracking  Tools  a  few  people 

video  teleconferencing 

Available  in  one  of  the  two  buildings  Used  often  by  some,  not  at  all  by  others 

document  imaging 

Available  Not  used 

EDI 

None 

CAD/CAM 

None 

Systems  Profile,  SPO  #4 


Cultural  dimension 

Key  features 

Comment 

Type  of  organization 

Mission 

Size 

Fragmentation 

Single  program  SPO 

Acquisition 
c.  600 

Divided  between  2  buildings;  some  TDY 

Across  the  street 

Turbulence 

Creation  of  IPT's;  job  description  changes 

Implementation  process 

No  user  input;  history  with  one  failure; 
most  people  informed  of  plans 

Computer  support  seen  as  champs; 
large  &  diverse  computer  support  group. 

Assumptions  re  computing 
Prestige 

Security 

Some  resistance  in  culture 

Moderate  due  to  airframe  age 

Little  classified  data 

Cultural  Profile,  Organization  4 
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SPO  #5  ~  Out  in  the  field 


Application 

Systems  used 

Comment 

e-mail 

Beyond  Mail 

Changing  to  Beyond  Mail  from  the  VAX 
All-in-One 

shared  databases 

AWDS;  Parts  Master;  FIN  Plan; 

All  used  by  very  few  people  in  the 

ALMANAC; 

organization;  2  used  by  single  Individuals 

workflow  tools 

Shared  Drives;  ACCESS;  HERBS 

Shared  Drives  used  by  about  fifty 

(Hanscom  Electronic  RFP  Bulletin 

percent  of  organization;  others  used  by 

Board);  AF  Forms;  CaLANder;  ACE-IT 
(Auto  Cost  Estimating-Integrated  Tools) 

less  than  twenty  percent  of  organization 

video  teleconferencing 

Available 

Newly  installed;  not  used 

document  imaging 

None 

EDI 

None 

CAD/CAM 

None 

Systems  Profile,  SPO  #5 


Cultural  dimension 

Key  features 

Comment 

Type  of  organization 

Mission 

Size 

Fragmentation 

Basket  SPO 

Acquisition 
c.  350 

Divided  between  2  buildings;  some  TDY 

One  building  off-base 

Turbulence 

Creation  of  IPT's;  downsizing;  leadership 

changes 

Implementation  process 

No  user  input;  history  of  problems  w/ 

Computer  support  not  viewed  as 

Assumptions  re  computing 
Prestige 

Security 

implementation  efforts 

Some  resistance  In  culture 

Moderate 

Little  classified  data 

champs;computer  support^  2 
contractors;  and  not  knowledgeable  In 
system  usage 

Cultural  Profile,  Organization  5 
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SPO  #6  ~  Eyes  in  the  skies 


Application 

Systems  used 

Comment 

e-mail 

All-in-One 

shared  databases 

LSAR;  Grumman’s  ISA;  LCC  (Life 
Cycle  Cost)  d-base 

All  three  used  by  one  individual  only 

workflow  tools 

Shared  Drives;  ACCESS;  ACE-IT  (Auto 
Cost  Est.  Integr  Tools);  CONTEXT; 

HARE  (Hanscom  Auto  RFP);  FIN  Plan; 
File  Maker  Pro;  HERBS  (Hanscom  Elect 
RFP  Bulletin  Board);  CASE  (Comp. 

Auto  Software  Eng) 

video  teleconferencing 

Available 

Used  frequently  by  some 

document  imaging 

Available 

Seldom  used 

EDI 

None 

CAD/CAM 

None 

System  Profile,  SPO  #6 


Cultural  dimension 

Key  features 

Comment 

Type  of  organization 

Single  program  SPO 

Mission 

Acquisition 

Size 

c.  550 

Fragmentation 

Divided  between  2  buildings;  remote 

One  building  off-base;  people  In  other 

locations  present;  frequent  TDY 

state 

Turbulence 

Creation  of  IPT's;  leadership  changes 

Implementation  process 

User  input  solicited;  rapid 

Computer  support  seen  as  champ  and 

implementation;  no  history  of  failures 

knowledgeable  in  system  used; 
computer  support  group  Is  large  & 
diverse 

Assumptions  re  computing 

receptive  culture 

Prestige 

Prestigious 

Security 

Some  classified  data 

Cultural  Profile,  Organization  6 
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Program  #7  —  The  orphan  program 


Application 

Systems  used 

Comment 

e-mail 

Lotus  Notes  E-mail 

shared  databases 

D043;  On  Lotus  Notes:  Open  Projects; 

D043  used  by  two  people;  use  of  Open 

Closed  Projects;  Points  of  Contact; 

and  Closed  Projects  was  mandatory  for 

Internal  Info;  Customer  Feedback; 

all  organizational  members;  five  used  by 

Address  Book;  Trip  REports;  Leave  and 
Events;  Awards;  Problem  Documentation 
Sheet 

a  majority;  three  used  by  a  minority 

workflow  tools 

Shared  Drives;  ITHINK;  WAR  (Weekly 

Shared  drives  and  Perform  used  by  a 

Activity  Rpts);  Perform 

majority 

video  teleconferencing 

None 

document  imaging 

None 

EDI 

None 

CAD/CAM 

None 

Systems  Profile,  Program  #7 


Cultural  dimension 

Key  features 

Comment 

Type  of  organization 

Basket  division  of  basket  SPO 

Autonomous  from  SPO 

Mission 

Sustainment 

Size 

c.  45 

Fragmentation 

All  in  one  building;  frequent  TOY 

ii 

Turbulence 

Creation  of  IPTs;  building  location  changes;  downsizing=personnel  loss 

Implementation  process 

User  input  solicited;  rapid 

Computer  support  seen  as  champ  and 

implementation;  no  history  of  failures; 
mandatory  usage  &  training 

knowledgeable  in  system  usage 

Assumptions  re  computing 

receptive  culture 

Prestige 

Low 

Security 

No  classified  data 

Cultural  Profile,  Organization  7 
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SPO  #8  —  Iron  on  target 


Application 

Systems  used 

Comment 

e-mail 

All-in  One 

shared  databases 

C-PAS  Mapper;  SPO-MIS  2;  SLIC  (Lys 

C-PAS  used  by  2  people;  others  used  by 

Logs  integr  Capab) 

single  Individuals 

workflow  tools 

Shared  Drives;  Project  Manage 

Shared  Drives  new,  used  by  a  minority; 
other  used  by  2  people 

video  teleconferencing 

Available 

used  infrequently 

document  imaging 

None 

EDI 

None 

CAD/CAM 

Systems  Profile,  SPO  #8 


Culturai  dimension 

Key  features 

Comment 

Type  of  organization 

Basket  SPO 

Autonomous  divisions 

Mission 

Acquisitions 

Size 

c.  400 

Fragmentation 

All  in  one  building;  some  TDY 

Turbulence 

Creation  of  iPT's;  downsizing=personnel  loss 

Implementation  process 

No  user  input;  slow  implementation 

Computer  support  viewed  as  champ; 
computer  support  group  is  large  and 

diverse  and  new 

Assumptions  re  computing 

Resistant  culture 

Prestige 

Intermediate 

Security 

Little  classified  data 

Cultural  Profile,  Organization  8 
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SPO  #9  —  Living  in  a  fishbowl 


Application 

Systems  used 

Comment 

e-mail 

Base  MIS  system 

Installing  Eudora  support  (GUI  system) 

shared  databases 

TOM  (Tech  Order  Module);  DAMIS 
(Depot  Acq.  MIS);  PMS  (Procurement 
Mgmt  System);  CDMS  (Contract  Data 
Mgmt  Sys):  FAR  (Fed  Acq  Regs); 
CLASS  (Consolidated  Logisitcal  Anal 
Sppt  Sys);  SLIC  (System  &  Logs  Integr 
Capabi!ity);TOCU  (Tech  Order  Control 
Sys) 

CLASS  and  TOCU  are  read  only 
databases  of  the  prime  contractor;  most 
of  these  databases  used  by  Logistics 
and  Acquistion  branch;  PMS  and  FAR 
used  by  contract  group;  CDMS  used  by 
Configuration  and  Data  branch 

workflow  tools 

Shared  Drives;  Formal  Mail 

Each  division  has  own  shared  drive;  2 
shared  drives  are  SPO  wide'  Formail 

Mail  used  by  majority 

video  teleconferencing 

Available 

Used  infrequently 

document  imaging 

EDI 

CAD/CAM 

Available 

None 

None 

Used  by  one  division 

Systems  Profile,  SPO  #9 


Cultural  dimension 

Key  features 

Comment 

Type  of  organization 

Single  Program  SPO 

Mission 

Acquisitions 

Size 

c.  250 

Fragmentation 

All  in  one  building;  frequent  TDY 

Turbulence 

New  building;  increase  in  personnel 

new  Big  dollar  program  under  Congressional 

leaders;  program  in  jeopardy 

pressure  for  accountability 

Implementation  process 

User  input  solicited 

Computer  support  viewed  as  champ; 
computer  support  group  is  large  and 
divese 

Assumptions  re  computing 

Receptive  culture 

Prestige 

Low 

Due  to  Congressional  investigation 

Security 

Little  classified  data 

Cultural  Profile,  Organization  9 


In  sum,  the  nine  systems  and  cultural  profiles  presented  here  underlie  the  knowledge  embedded  in  the 
FRAMEAVORK  tool.  This  overview  of  the  data  is  presented  as  a  guide  to  further  understanding  of 
the  relationships  between  organizational  culture  and  systems  implementation. 
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